[jira] [Commented] (PIG-4538) Pig script fail with CNF in follow up MR job

["Daniel Dai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533543#comment-14533543 ] 

Daniel Dai commented on PIG-4538:
---------------------------------

This is actually introduced by FetchOptimizer (PIG-3642). Even if we are using MR engine eventually, Pig still went through FetchOptimizer. And FetchOptimizer:179 set the whole physical plan to all operators, so the LoadFunc get serialized into the conf. And since Pig doesn't ship LoadFunc to backend in the followup MR job, Pig fails with CNF exception.

> Pig script fail with CNF in follow up MR job
> --------------------------------------------
>
>                 Key: PIG-4538
>                 URL: https://issues.apache.org/jira/browse/PIG-4538
>             Project: Pig
>          Issue Type: Bug
>          Components: tez
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.15.0
>
>
> The following script fail:
> {code}
> in1 = LOAD 'data.txt' AS (ident:chararray);
> in2 = LOAD 'data.txt' USING org.apache.pig.piggybank.storage.CSVExcelStorage(',') AS (ident:chararray);
> in3 = LOAD 'data.txt';
> joined = JOIN in1 BY ident LEFT OUTER, in2 BY ident;
> crossed = CROSS joined, in2;
> DUMP crossed;
> {code}
> The third MR job fail with message:
> ERROR 1070: Could not resolve org.apache.pig.piggybank.storage.CSVExcelStorage using imports: \[, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.\]
> This is a regression from Pig 0.13, possibly introduced by PIG-3591.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)