[scxml] Re: Re: Re: our project setup: any tips specifically on performance/speed?

["R.C. Hoekstra" <r....@erasmusmc.nl>]

Hi Ate, hi list, 

I can share some more information now, and post some code. 

First of all, on our project: 

No, we don't use threads at the moment. Multithreading seems to become a nightmare with over 100 k engine instances. 

We also don't use the <datamodel> tags at the moment. As datamodel is still in the planning according to the roadmap, I found it a bit dangerous to rely on it. Also, it does seem a bit inconvenient for our purpose, but if that's not the case I'm happy to become convinced of the contrary.

(De)Serialization is not an issue at the moment, though it might become an issue in the future. We're considering two strategies: Maybe just log every transition and do analysis on that log with some tool. Or dump the whole population of 100,000 agents / scxmlExecutor instances every now and then at fixed times in some database structure for further analysis. We are still open to suggestions, what this concerns, though that is of course not really an scxml issue. 
For the rest: the simulation is just running in memory, and for now that doesn't seem to be a problem for the hardware. 

So here are some key parts of our code / setup (Ate, I can post them without problems now). 
Main question is: which are the elements which are possibly a bottleneck for the performance, and are there any alternatives for these elements? 


So the disease is modelled by an human.scxml file which defines the states of the disease. Example:

 	<state id="asymptomatic">
		<onentry>
                        <script>
                            agent.infectivity = 0.2
                        </script>
			<ntd:schedule event="from.asymptomatic" distr="EXPONENTIAL" mean="72 d" chances="0.01"/>
		</onentry>
		<transition event="from.asymptomatic" cond="_event.data == 0" target="symptomatic"/>  			
		<transition event="from.asymptomatic" cond="_event.data == 1" target="fullyRecovered"/>   			
  	</state>

The ntd:schedule tag is responsible for scheduling the transitions in our EventManager. The Action class responsible for this uses the attributes to draw a random number and schedules the passed event with the resulting time in our own eventManager. When that scheduled time has come, the eventManager sends the event back to the state machine instance in order to invoke the transition.
The chances attribute defines the chances for each transition defined: it is a space separated list of doubles indicating transition chances. In this case the first transition should get a chance of 0.01 of happening, and as there is no other number given, the final remaining transition gets a chance of 0.99 to happen. The index number of the transition is passed back via the payload and tested in the cond attribute of the transition. 

The script tag is responsible for setting agent properties - in this case the infectivity. Each agent has an own SCXMLExecutor instance, and sets itself to the executor's rootcontext, so it can be accessed in the scxml files. Each agent is also its own scxml Listener.
All is single threaded. I think the EventManager is smart enough to handle and store 100,000 or more scheduled events. The EventManager is passed to our own implementation of Evaluator, which is simply a child class of JexlEvaluator, but with awareness of math (statistic distributions & random number generator), eventManager and the time running in the simulation. 

Then, we also need to model treatments. So we have one main scxml file with the following content.

	<parallel id="alive">
		<onentry>
                        <!-- this schedules natural death... --> 
			<ntd:schedule event="die.natural" distr="EXPONENTIAL" mean="14600.0 d" />
		</onentry>
		<transition event="die.natural" target="death" />
		
		<state id="biology" src="VL_human.scxml" />
		<state id="treatments" src="VL_treatments.scxml" />
	</parallel>

	<final id="death"/>

So biology definitions define the disease itself, and all its states; treatments define how patients are treated: subjects will pass states "untreated", "tested" and various possible treatments. As treatments can happen in various states of the disease, I though it best to use parallel states here. A treatment raises events which in "biology" will cause the patient to recover. 

I hope this is enough to give a rough picture, and I hope you guys could point out some performance bottlenecks. I understand however that most performance bottlenecks are expected to be in the IO/(de)serialization part. 
I could post some more code snippets if that is helpful or interesting.


thanks, best regards, Rinke

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org