[GitHub] johnyangk opened a new pull request #38: [NEMO-48] Do not merge broadcasted data into a single partition

[GitBox <gi...@apache.org>]

johnyangk opened a new pull request #38: [NEMO-48] Do not merge broadcasted data into a single partition
URL: https://github.com/apache/incubator-nemo/pull/38
 
 
   JIRA: [NEMO-48: Do not merge broadcasted data into a single partition](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-48)
   
   **Major changes:**
   - In NemoPipelineVisitor, set edge to Broadcast only when the dst vertex is CreateViewTransform (and not also when the src vertex is CreateViewTransform)
   - This allows CreateViewTransform to be pipelined with downstream vertices, and fetch brodcasted data in parallel
   
   **Minor changes to note:**
   - In executionPropertyCheck, allow for non-broadcast SideInputs
   - Pass down intra-stage isSideInput in PhysicalPlanGenerator
   - A better way to set isFromSideInput in DataFetchers
   
   **Tests for the changes:**
   - N/A (no new feature, existing ITCases pass)
   
   **Other comments:**
   - N/A
   
   resolves [NEMO-48](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-48)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services