[jira] [Updated] (PIG-3657) New partition filter extractor fails with NPE

["Cheolsoo Park (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Cheolsoo Park updated PIG-3657:
-------------------------------

    Attachment: PIG-3657-1.patch

The patch includes a unit test that demonstrates the issue. Without the fix, the test throws a NPE.

Regarding the cause, the filtered plan returned by FilterExtrator is not always translatable to physical plan because newly added OR/AND expressions do not copy successors.

For the aforementioned example, FilterExtractor returns the following filtered plan-
{code}
(Name: Or Type: null Uid: null)
|
|---(Name: LessThan Type: boolean Uid: 112)
|
|---(Name: Equal Type: null Uid: null)
    |
    |---(Name: Project Type: null Uid: null Input: 0 Column: 42)
    |
    |---(Name: Constant Type: null Uid: null)
{code}
As can be seen, LessThan has no children resulting in NPE while converting it to physical plan.

> New partition filter extractor fails with NPE
> ---------------------------------------------
>
>                 Key: PIG-3657
>                 URL: https://issues.apache.org/jira/browse/PIG-3657
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>             Fix For: 0.13.0
>
>         Attachments: PIG-3657-1.patch
>
>
> To reproduce the issue, try a filter expression as follows-
> {code}
> b = FILTER a BY partition_column < x OR (partition_column == y AND non-partition_column == z); 
> {code}
> Pig fails with NPE, and the stack trace is something like this-
> {code}
> Caused by: java.lang.NullPointerException
>     at org.apache.pig.newplan.FilterExtractor.checkPushDown(FilterExtractor.java:252)
>     at org.apache.pig.newplan.FilterExtractor.visit(FilterExtractor.java:115)
>     at org.apache.pig.newplan.logical.rules.PartitionFilterOptimizer$PartitionFilterPushDownTransformer.transform(PartitionFilterOptimizer.java:150)
>     at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:110)
>     ... 18 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)