[GitHub] jeongyooneo opened a new pull request #2: [NEMO-7] Intra-TaskGroup pipelining

[GitBox <gi...@apache.org>]

jeongyooneo opened a new pull request #2: [NEMO-7] Intra-TaskGroup pipelining
URL: https://github.com/apache/incubator-nemo/pull/2
 
 
   * **Refactored `TaskGroupExecutor` to a pipelined execution model(Thanks to @sanha)**
      * Per-`Task` `InputReader`/`OutputWriter` and `TaskState` management are removed.
      * Per-`Task` `Pipe` acts as a local writer for this `Task` to where it emits output, and a local reader for its children `Task`s from where they receives input.
      * Input is deserialized element-wise, trickle down to the end of `TaskGroup` DAG in a recursive manner. 
          * `iteratorIdToDstTasksMap` maintains source input(data from bounded source or other `TaskGroup`s) to the `Task` DAG that processes this input.
          * `pipeToDstTasksMap` maintains intra-TaskGroup input to the `Task` DAG that processes this input.
   
   <p align="center">
   <img src="https://user-images.githubusercontent.com/6097406/36969167-11bdbb90-20a8-11e8-89f8-d3d1c459154c.png" width="400">
   </p>
   
   <p align="center">
   <img src="https://user-images.githubusercontent.com/6097406/36969232-4517e236-20a8-11e8-93e2-f4e8d8ae8f84.png" width="400">
   </p>
   
   JIRA: [NEMO-7](https://issues.apache.org/jira/browse/NEMO-7)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services