[GitHub] bjustin-ibm opened a new pull request #178: convert from threading to multiprocessing

[gi...@git.apache.org]

bjustin-ibm opened a new pull request #178: convert from threading to multiprocessing
URL: https://github.com/apache/incubator-openwhisk-package-kafka/pull/178
 
 
   Despite having a threading API, the Python GIL ensures that no Python code in the same process is allowed to execute in parallel. Threading, it seems, is a tool to work with concurrency without real parallel execution :( However, the `multiprocessing` module allows for true parallel execution by spawning separate Python processes. 
   
   In order to accommodate more and more consumers, I have ported this over to use the `multiprocessing` module which, thankfully, has pretty much the same API as `threading`. Interprocess communication is accomplished by using shared dictionaries that can be read/written from the main process and the child consumer processes. Each shared dictionary conveys the current status of a single consumer, as well as serves as a vehicle for requesting state changes to the child process by simply setting the `desiredState` key in the shared dictionary.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services