[jira] [Updated] (PIG-4574) Eliminate identity vertex for order by and skewed join right after LOAD

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rohini Palaniswamy updated PIG-4574:
------------------------------------
    Attachment: PIG-4574-fix-testfailures.patch

Fixing unit test failures in https://builds.apache.org/job/Pig-trunk-commit/2232/testReport/

  - TestPigServerLocal.testSkipParseInRegisterForBatch
For LOAD + ORDER BY, since we repeat the LOAD separately for sampler and partitioner like in mapreduce, there is increase in the number of LoadFunc instantiations.
  - TestTezGraceParallelism.testJoinWithDifferentDepth
scope id change due to different plan for LOAD + ORDER BY


> Eliminate identity vertex for order by and skewed join right after LOAD
> -----------------------------------------------------------------------
>
>                 Key: PIG-4574
>                 URL: https://issues.apache.org/jira/browse/PIG-4574
>             Project: Pig
>          Issue Type: Sub-task
>          Components: tez
>            Reporter: Rohini Palaniswamy
>            Assignee: Rohini Palaniswamy
>             Fix For: 0.16.0
>
>         Attachments: PIG-4574-1.patch, PIG-4574-fix-testfailures.patch
>
>
> If ORDER BY or SKEWED JOIN is the operator immediately following LOAD+FOREACH without any FILTER, then data should be read again from HDFS for the partitioner vertex instead of writing to a identity vertex and reading from it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)