Direct Call to CompuCell3D from Python
In a typical situation, you will most often run CC3D using GUI. You create simulation, run it, look at the results, modify parameters, run again and the process continues. But what if you would like to be more “methodical” in finding optimal parameters? CC3D comes with a convenience module that allows you to directly call CC3D as a Python function, pass information to your simulations, and get return value(s) from them. As you can tell already, functionality like this allows you to use CC3D as a black box that takes inputs and returns outputs in advanced, integrated workflows (e.g., integration with various optimization packages to find desired set of parameters for your simulation).
How does it work?
Note
All examples from this section can be found in CompuCell3D/core/Demos/CallableCC3D
Step 1
If you want to call CC3D engine from a Python function, pass information to it and/or get information from it, then you need to modify your existing simulation to process incoming information and/or return outgoing information. This is a very easy step - take a look at the code below:
from cc3d.core.PySteppables import *
from random import random
class CellsortSteppable(SteppableBasePy):
def __init__(self, frequency=10):
SteppableBasePy.__init__(self, frequency)
def start(self):
input_val = self.external_input
def step(self, mcs):
if mcs == 100:
self.external_output = 200.0 + random()
This is a “regular” steppable. There are two new parts of interest. The first new part is where we retrieve a value
from external_input
. The value of this property is set in Python before CC3D is called and is
passed to CC3D during instantiation of a simulation through a Python function. The second new part is the line where we
modify external_output
. This property that will persist even after simulation is
finished is the way we set what is returned by our simulation. Here, our return value is set to be a sum of number
200.0
and a random number between 0 and 1. This return value can be set at any point in the simulation. In
particular it often makes sense to set it in the finish
function, but for illustration purposes we set it in
step
function.
Step 2
Once we have a simulation we need to write a Python script from which we will call our value-returning simulations. We will first show a minimal example where we call CC3D a few times from a Python script, pass a value to each simulation and store the return value of each simulation in a list. Here is the code:
1from os.path import dirname, join, expanduser
2from cc3d.CompuCellSetup.CC3DCaller import CC3DCaller
3
4
5def main():
6
7 number_of_runs = 4
8
9 # You may put a direct path to a simulation of your choice here and comment out simulation_fname line below
10 # simulation_fname = <direct path your simulation>
11 simulation_fname = join(dirname(dirname(__file__)), 'cellsort_2D', 'cellsort_2D.cc3d')
12 root_output_folder = join(expanduser('~'), 'CC3DCallerOutput')
13
14 # this creates a list of simulation file names where simulation_fname is repeated number_of_runs times
15 # you can create a list of different simulations if you want.
16 sim_fnames = [simulation_fname] * number_of_runs
17
18 ret_values = []
19 for i, sim_fname in enumerate(sim_fnames):
20 cc3d_caller = CC3DCaller(
21 cc3d_sim_fname=sim_fname,
22 screenshot_output_frequency=10,
23 output_dir=join(root_output_folder,f'cellsort_{i}'),
24 sim_input=i,
25 result_identifier_tag=i
26 )
27
28 ret_value = cc3d_caller.run()
29 ret_values.append(ret_value)
30
31 print('return values', ret_values)
32
33
34if __name__ == '__main__':
35 main()
In line 1 we import functions from os.path
package that will be used to create paths to files. In line 2 we
import CC3DCaller
class. CC3DCaller
object runs single simulation and returns simulation return value.
Note
Simulation return value is a dictionary. This allows for quite a lot of flexibility. In particular, you are not limited to a single return value but can use multiple return values.
In line 11 we construct a path to a simulation that takes inputs and returns a value. This is a simulation that is
bundled with CC3D. If you want to run different simulation you would replace code in line 11 with a direct path to your
simulation. Line 12 defines location where we will write simulation output files (think of it as custom version of
CC3DWorkspace
folder that CC3D normally uses for simulation output).
Note
When you rerun your multiple simulations using script above you may want to make sure that simulation output folders are empty to avoid overwriting output from previous runs
In line 16 we construct a list of simulations we want to run. Notice that we use Pythonic syntax to create a list with
multiple copies of the same element. [simulation_fname] * number_of_runs
constructs a list where simulation_fname
is repeated number_of_runs
times.
In line 18 we create a list that will hold results. Line 19 starts a loop where we iterate over the simulation paths we
stored in the list sim_fnames
.
In line 20 we create CC3DCaller
object where we pass simulation name, screenshot output frequency, output directory
for this specific simulation, input to the simulation and a tag (identifier) that is used to identify return results.
In our case we use integer number i
as an input and identifier but you can be more creative. In general,
whatever is passed to the keyword argument sim_input
is available to our steppables as the property
external_input
, and whatever we set the steppable property
external_output
to in our steppables will be returned. Finally in line 28 we
execute simulation and get return value of the simulation and in line 29 is appended to ret_values
. Line 31 prints
return values.
If we run this script the output of print statement in line 31 will look something like (because we use random()
function we do not know exact outputs):
return values [{'tag': 0, 'result': 200.8033875687598}, {'tag': 1, 'result': 200.6628249954859}, {'tag': 2, 'result': 200.6617630355885}, {'tag': 3, 'result': 200.30450775355195}]
Notice that a single simulation returns a dictionary as a result. For example simulation with tag 1
returned
{'tag': 1, 'result': 200.6628249954859}
. By “consuming” this dictionary in Python we can extract identifier using
ret_values[1]['tag']
syntax and if we want to get the result we would use ret_values[1]['result']
.
Notice that in this example ret_values[1]['result']
is a floating point number but you can write your simulation
in such a way that the result can be another dictionary where you could return multiple values.
Applications
We mentioned it at the beginning, but the examples we are showing here are only to illustrate a technique of how to
call CC3D engine from Python script. Executing several simulations inside a Python loop is not that exciting but
coupling it to an optimization algorithm or sensitivity analysis script is actually more practical. We include a
simple example of integrating CC3D with the SciPy optimization module to do model calibration in
CompuCell3D/core/Demos/CallableCC3D/ChemotaxisCalibrateDemo
.
Step 3
In order to run above script you need to set up the environment that is shipped with the CC3D binary distribution or the one that you used to compile CC3D. Let’s get started. We will walk you through the steps necessary to run the above scripts on various platforms.
Note
Users who installed CC3D directly from conda only need to activate the conda environment in which CC3D is installed to configure the environment from a terminal.
Let’s start with windows.
Windows
Open a terminal in the root directory of the CC3D installation and issue call
on the script conda-shell.bat
,
call conda-shell
The environment is now configured to execute CC3D in Python.
Next I navigate to location where my script from Step 2
is installed
cd c:\CompuCell3D-py3-64bit\Demos\CC3DCaller\cc3d_call_single_cpu
I replace the line 11 of the script from Step 2
simulation_fname = join(dirname(dirname(__file__)), 'cellsort_2D', 'cellsort_2D.cc3d')
with
simulation_fname = r'c:\CompuCell3D-py3-64bit\Demos\CallableCC3D\cellsort_2D\cellsort_2D.cc3d'
The reason I am doing it because in real application you probably have to do it anyway. You specify directly what simulation you want to run
Finally, in the console I execute the following:
python cc3d_call_single_cpu.py
Make sure that you are in the correct directory when you run the last command.
Linux
Running simulation on Linux is very similar to running on Windows. We start by opening a terminal in the
root directory of the CC3D installation and calling source
on conda-shell.sh
,
source conda-shell.sh
Next, I go to /home/m/411_auto/Demos/CC3DCaller\cc3d_call_single_cpu
and execute the script from Step 2
.
It is also useful to change line 11 of the script from
simulation_fname = join(dirname(dirname(__file__)), 'cellsort_2D', 'cellsort_2D.cc3d')
to
simulation_fname = '/home/m/411_auto/Demos/CC3DCaller/cellsort_2D/cellsort_2D.cc3d'
cd /home/m/411_auto/Demos/CC3DCaller/cc3d_call_single_cpu
python cc3d_call_single_cpu.py
In your output you should see the following lines
return values [{'tag': 0, 'result': 200.8033875687598}, {'tag': 1, 'result': 200.6628249954859}, {'tag': 2, 'result': 200.6617630355885}, {'tag': 3, 'result': 200.30450775355195}]
Mac
To run script from Step 2
you would follow described in the Linux section above. The only difference is that
you will be using conda-shell.command
environment variable setter script instead of
conda-shell.sh
. As before, all you need to run is source conda-shell.command
to set up the environment
and modify script from Step 2
to include direct path to the simulation.