You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Praveen Rachabattuni (JIRA)" <ji...@apache.org> on 2015/02/24 04:51:50 UTC

[jira] [Updated] (PIG-4239) "pig.output.lazy" not works in spark mode

     [ https://issues.apache.org/jira/browse/PIG-4239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Praveen Rachabattuni updated PIG-4239:
--------------------------------------
    Fix Version/s: spark-branch

> "pig.output.lazy" not works in spark mode
> -----------------------------------------
>
>                 Key: PIG-4239
>                 URL: https://issues.apache.org/jira/browse/PIG-4239
>             Project: Pig
>          Issue Type: Bug
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>             Fix For: spark-branch
>
>         Attachments: PIG-4239.patch, PIG-4239_1.patch, lazy, lazy.pig
>
>
> If "pig.output.lazy" is "true", empty part files from the output will be omitted.
> steps to reproduce
> 1. set "pig.output.lazy" as “true" in $PIG_HOME/conf/pig.properties.
> 2. run following lazy.pig script in spark mode:
> cat lazy.pig
> a = load '/user/pig/lazy' using PigStorage();
> b = filter a by $0  == 'hey';
> c = store b into '/tmp/lazy.out';
> lazy.pig and lazy are attached
> 3.  empty file "/tmp/lazy.out/part-rxxxx” is still generated, the empty file is expected not to be generated when "pig.output.lazy" is "true"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)