Parameter Scans¶
When building biomedical simulations it is a common practice to explore parameter space to search for optimal solution or to study the robustness of parameter set at hand. In the past researchers have used (or abused) Python to run multiple replicas of the same simulation with different parameter set for each run. Because this approach usually involved writing some kind of Python wrapper on top of existing CC3D code, more often than not it led to hard-to-understand codes which were difficult to share and were hard to access by non-programmers.
Current version of CC3D attempts to solve these issues by offering users ability to create and run parameter scans directly from CC3D GUI’s or from command line. The way in which parameter scan simulation is run is exactly the same as for “regular”, single-run simulation.
To describe parameter scan users use Twedit++ to generate an XML file which describes the parameter scan. This XML file is referenced from .cc3d file (see example below) and this is how CC3D figures out that it has to scan parameter space:
<Simulation version="3.5.1">
<XMLScript Type="XMLScript">Simulation/CellSorting.xml</XMLScript>
<PythonScript Type="PythonScript">Simulation/CellSorting.py</PythonScript>
<Resource Type="Python">Simulation/CellSortingSteppables.py</Resource>
<ParameterScan Type="ParameterScan">Simulation/ParameterScanSpecs.xml</ParameterScan>
</Simulation>
As compared to “regular” .cc3d
file the one we show above has extra line
with ParameterScan XML element.
Clearly if we comment this line we can run the simulation in a single-run mode. This is great advantage of introducing additional XML file for parameter scans – the simulation code (CC3DML, Python scripts) is unmodified and can be run as-is in case modelers decide to turn off scanning of the parameter space.
Setting up Parameter Scan Using Twedit++¶
To setup up parameter scan we open any valid .cc3d project file in Twedit++:
We right-click on the project and select Add Parameter Scan
option:
Notice that Parameter Scan
appears as a part of the .cc3d
project. To
add a parameter from the CC3DML to the parameter scan we right-click on
the CC3DML file and select Open Scan Editor
:
Notice that Twedit++ opens up a new tab called Parameter Scan Tmp File
.
This is read-only file that you use to select parameters for scanning
purposes. To do that, you click in the desired place of this file. For
example if you want to to run simulation with different Temperature
parameters you click in the line with Temperature
parameter and then
righ-click to get access to Add To Scan…
option:
After you choose this option Twedit++ displays parameter scan configuration dialog:
This dialog displays all parameters in the given XML element which can change. In our case of a simple XML element we have only oneway to change the element and it is through its value (10). When we click edit we get to next dialog
Where we fill minimum, maximum value, select value type (we picked
integer but we could have picked float as well) and generate values
(given by the Number Of Steps
) according to given distribution (here we
picked linear which means we generate 5 equally spaced values between 2
and 20 inclusive). When we open parameter scan XML file we see that our
choices made in the GUI got translated into XML format:
To add more parameters we position the cursor in the desired location in
the Parameter Scan Tmp File
tab, right click to select Add To Scan…
option and follow steps outlined above. When we select more complicated
element e.g. <Dimensions>
we will get the following parameter
configuration dialog:
Notice that for this element there are 3 values which can change when we
pick , say y
, and generate values in the next pop-up widget we will end
up with the following parameter scan XML file:
As you can tell this XML file gets updated automatically so users do not need to type XML file.
Adding Python parameters to parameter scan is even easier. We open up Python file using Open Scan Editor option:
Later we go to the Parameter Scan Tmp File
tab which now has Python code
open in the read-only mode and right click on any global variable to add
it to the parameter scan. We have only one global variable defined in
our demo script (MYVAR
) so in the entire script only one line which
defines MYVAR
can be used to ope up Parameter scan dialog:
As a result Generate Parameter Values
dialog will pop up and we use it
in exactly the same way we used it for the XML parameters:
After this step our parameter scan XML file looks as follows:
Finally we can also edit the output directory of the parameter scan results by manually editing the parameter scan XML. Here we set it to
<OutputDirectory>InfoPrinter\_ParameterScan</OutputDirectory>
Running Parameter Scans¶
As you can see defining parameters scans is quite easy. Running simulation is easy as well and you have two options. The most straightforward (but probably not the best – keep reading to find the recommended way of running) is to open up the project in the Player and run it:
Notice that the title bar of the player informs you that you are running
parameter scan . It also displays number of the parameter simulation
that is currently being executed. (Iteration: 2
). The results of the
parameter scans are written to the Simulation Output
directory :
In my case it is c:/Users/m/CC3DWorkspace
, the default setting for
simulation output directory is is <your home directory>/CC3DWorkspace
.
When we look into this directory we will see that it contains subdirectories corresponding to the consecutive simulation runs – each such simulation corresponds to a different set of parameters:
Since at the moment CC3D is performing exhaustive parameter scan you have to be careful with the number of parameters you scan and number of values each parameter can assume. Simply put the total number of simulation to run when doing exhaustive parameter scan is equal to a producs of number of values of each parameter. Scanning 10 parameters with 10 values each will require 10 billion simulations. This is a large number and it will take quite a lot of time before all results are ready.
Important: When parameter scan runs it modifies ParameterScan.xml
file. In particular it record current iteration for each parameter. To
rerun parameter scan again from the beginning you need to set
CurrentIteration values for each parameter to 0. You can do it manually
or use Twedit++ option Reset Parameter Scan:
In a nutshell this is all it takes to run parameter scan in CC3D 3.7.1. We will add more options to this feature but at least now you can probe your parameter scans without writing clumsy-looking Python wrappers.
The recommended way of running parameter scan is via script called
paramScan.sh . paramScan.sh takes same command line arguments as
compucell3d.sh
or runScript.sh
(notice on windows we use .bat
extension
and on OSX .command
). Here is the syntax:
paramScan.sh –i <cc3d project file> --guiScan –maxNumberOfRuns=20 [remaining command line options used in ``compucell3d.sh`` or ``runScript.sh``]
--guiScan
– will ensure that paramScan.sh
will be run using Player. If
you do not use this option the parameter scan will internally use
runScrip.sh
, hence no gui, which is often preferred way of running
multiple jobs on clusters. With -–guiScan
enabled paramScan.sh
calls
compucell3d.sh
internally so you can pass any options you would normally
use with compucell3d.sh
. When –-guiScan
is disabled paramScan.sh
calls
runScript.sh
so you any options you would normally use with
runScript.sh
.
--maxNumberOfRuns
- using this option you can stop parameter scan after
given number of simulations. By default the parameter scan will run
until all the simulation have been finished. This option is most helpful
during debugging stage
[remaining command line options used in compucell3d.sh or runScript.sh]
– here you simply pass additional options you would use with - see
explanation of the -–guiScan
switch for explanation
Example commands:¶
paramScan.sh –i vascular.cc3d --noOutput
paramScan.sh –i vascular.cc3d --guiScan
The benefit of using paramScan.sh
is that it is fault tolerant. LEt’s say,
your simulation crashes in the middle because e.g. somewhere in the
Python script you divide by 0. paramScan.sh
will handle this situation
and start new subsequent simulation. If you used Player or runScript.sh
directly to run parameter scan the parameter scan would simply stop in
that situation. When you do large runs on clusters you might want to
keep running scan even if some of the simulations run into trouble. This
is why using paramScan.sh is preferred way of running parameter scans,
starting with 3.7.3 version.
You may also find example command to run parameter scan in Twedit++:
CC3D Python->Parameter Scan Command Line
.
Parameter Scan Configuration Details¶
Let us now discuss how we describe parameter scan for parameters defined in the CC3DML file. In our example we will be scanning parameters defined in the Potts section:
<CompuCell3D version="3.6.2">
<Potts>
<Dimensions x="100" y="100" z="1"/>
<Steps>10</Steps>
<Temperature>10.0</Temperature>
<NeighborOrder>2</NeighborOrder>
</Potts>
Let us look into XML file describing parameter scan itself (note that you do not need to type this code manually – Twedit++ will generate it for you with just few clicks):
<ParameterScan version="3.7.0">
<OutputDirectory>CellSorting_ParameterScan</OutputDirectory>
<ParameterList Resource="Simulation/CellSorting.xml">
<Parameter CurrentIteration="0" Name="y" Type="XML_ATTR" ValueType="float">
[['CompuCell3D','version','3.6.2'],['Potts'],['Dimensions','x','100','y','100','z','1']]
<Values>65.0,110.0,120.0</Values>
</Parameter>
<Parameter CurrentIteration="0" Name="Steps" Type="XML_CDATA" ValueType="int">
[['CompuCell3D','version','3.6.2'],['Potts'],['Steps']]
<Values>1,2,3,4,5,6</Values>
</Parameter>
</ParameterList>
<ParameterList Resource="Simulation/CellSortingSteppables.py">
<Parameter CurrentIteration="0" Name="MYVAR" Type="PYTHON_GLOBAL" ValueType="int">
<Values>0,1,2</Values>
</Parameter>
<Parameter CurrentIteration="0" Name="MYVAR1" Type="PYTHON_GLOBAL" ValueType="string">
<Values>"abc1,abc2","abc"</Values>
</Parameter>
</ParameterList>
</ParameterScan>
<OutputDirectory>CellSorting_ParameterScan</OutputDirectory>
is the
first child of the root XML element (<ParameterScan version="3.7.0">
).
It specifies the name of the directory where CC3D will store the results
of the scan. As it is always the convention in CC3D this directory name
is relative to the simulation output path (default setting
<your_home_directory>/CC3DWorkspace
).
The next two elements are “container elements” (<ParameterList>
) where
we store description for each parameter we want to scan. We have
separate container element for each file in the .cc3d simulation. Here,
we scan parameters defined in two files Simulation/CellSorting.xml
and
Simulation/CellSortingSteppables.py
. Consequently we have two container
XML elements:
<ParameterList Resource="Simulation/CellSorting.xml">
and .. code-block:: xml
<ParameterList Resource=”Simulation/CellSortingSteppables.py”>
Each sub-element of <ParameterList>
is an element how one particular
parameter will be scanned. Let us concentrate first on the XML
parameters.
<Parameter CurrentIteration="1" Name="y" Type="XML_ATTR" ValueType="float">
[['CompuCell3D','version','3.6.2'],['Potts'],['Dimensions','x','100','y','100','z','1']]
<Values>65.0,110.0,120.0</Values>
</Parameter>
Each <Parameter>
element has CurrentIteration
attribute which determines
an index of the current value of the parameters. In this example this
index has value 1, hence it points to second element of the list
<Values>
. In this case the value of the parameter will be 110.0
. The
name of the parameter in the XML file which will assume value of 110.0
is y
as indicated by Name="y"
attribute. The parameter y
is an attribute
of the
``<Dimensions x="100" y="100" z="1"/>``
element of the CC3DML file as shown above.For this reason, in the
parameter scan XML file we set the type of this parameter to be
XML_ATTR
(Type="XML_ATTR"
) and we also set its value-type to be float
(ValueType="float"
). While dimensions are integer numbers we purposely
set it to float to demonstrate that it is up to the modeler to assign
correct value type in the <Parameter>
element. In this case no harm will
be done as the float will be converted to integer during parsing but in
general one has to be aware that specifying incorrect value type may
result in malfunctioning simulation.
The value of the <Parameter>
element:
[['CompuCell3D','version','3.6.2'],['Potts'],['Dimensions','x','100','y','100','z','1']]
Determines XML access path that allows CC3D to locate correct parameter
y in the CC3DML file. For details on how to construct access path please
check Steering section of this manual. The only change as compare to
steering is that we include ['CompuCell3D','version','3.6.2']
as a
root element and the entire access path is contains in list bracket
Python operators:
[[RootSpec],[Child1Spec],[Child1\_1 spec],…]
As compared to XML parameter scan specification, specifying Python parameter scan is much simpler. This is because In Python we only allow global parameters to be scanned. Consequently each of the <Parameter> elements will have structure similar to the one below:
<Parameter CurrentIteration="0" Name="MYVAR" Type="PYTHON_GLOBAL" ValueType="int">
<Values>0,1,2</Values>
</Parameter>
Here the parameter that we will scan has name MYVAR
, is of value-type
integer (int
) and its type is PYTHON_GLOBAL
.
For completeness we include Python code snippet which lists MYVAR
:
from PySteppables import *
import CompuCell
import sys
MYVAR = 10
MYVAR1 = 'new str'
class CellSortingSteppable(SteppableBasePy):
def __init__(self, _simulator, _frequency=1):
SteppableBasePy.__init__(self, _simulator, _frequency)
def step(self, mcs):
global MYVAR
print 'MYVAR=', MYVAR
for cell in self.cellList:
if cell.type == self.DARK:
cell.lambdaVecX = -0.5