You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2011/01/20 23:23:44 UTC
[jira] Created: (PIG-1814) mapred.output.compress in SET statement
does not work
mapred.output.compress in SET statement does not work
-----------------------------------------------------
Key: PIG-1814
URL: https://issues.apache.org/jira/browse/PIG-1814
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.8.0
Setting output compression using "SET" in the script does not work:
SET mapred.output.compress true;
SET mapred.output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
We did some trick to make individual compression setting for multistore work. Instead of the above parameter, using the following works:
SET output.compression.enabled true;
SET output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
However, this is against intuition. We should use mapred.output.compress/mapred.output.compression.codec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (PIG-1814) mapred.output.compress in SET statement
does not work
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1814:
----------------------------
Attachment: PIG-1814-1.patch
> mapred.output.compress in SET statement does not work
> -----------------------------------------------------
>
> Key: PIG-1814
> URL: https://issues.apache.org/jira/browse/PIG-1814
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: PIG-1814-1.patch
>
>
> Setting output compression using "SET" in the script does not work:
> SET mapred.output.compress true;
> SET mapred.output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> We did some trick to make individual compression setting for multistore work. Instead of the above parameter, using the following works:
> SET output.compression.enabled true;
> SET output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> However, this is against intuition. We should use mapred.output.compress/mapred.output.compression.codec.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-1814) mapred.output.compress in SET
statement does not work
Posted by "Xuefu Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13024971#comment-13024971 ]
Xuefu Zhang commented on PIG-1814:
----------------------------------
+1 patch looks good.
> mapred.output.compress in SET statement does not work
> -----------------------------------------------------
>
> Key: PIG-1814
> URL: https://issues.apache.org/jira/browse/PIG-1814
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: PIG-1814-1.patch
>
>
> Setting output compression using "SET" in the script does not work:
> SET mapred.output.compress true;
> SET mapred.output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> We did some trick to make individual compression setting for multistore work. Instead of the above parameter, using the following works:
> SET output.compression.enabled true;
> SET output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> However, this is against intuition. We should use mapred.output.compress/mapred.output.compression.codec.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-1814) mapred.output.compress in SET statement
does not work
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1814:
--------------------------------
Fix Version/s: (was: 0.8.0)
0.9.0
This issue has a very reasonable workaround - delaying till 0.9
> mapred.output.compress in SET statement does not work
> -----------------------------------------------------
>
> Key: PIG-1814
> URL: https://issues.apache.org/jira/browse/PIG-1814
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.9.0
>
>
> Setting output compression using "SET" in the script does not work:
> SET mapred.output.compress true;
> SET mapred.output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> We did some trick to make individual compression setting for multistore work. Instead of the above parameter, using the following works:
> SET output.compression.enabled true;
> SET output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> However, this is against intuition. We should use mapred.output.compress/mapred.output.compression.codec.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-1814) mapred.output.compress in SET
statement does not work
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13024853#comment-13024853 ]
jiraposter@reviews.apache.org commented on PIG-1814:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/661/
-----------------------------------------------------------
Review request for pig.
Summary
-------
See PIG-1814
This addresses bug PIG-1814.
https://issues.apache.org/jira/browse/PIG-1814
Diffs
-----
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/PigServer.java 1095577
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java 1095577
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestBZip.java 1095577
Diff: https://reviews.apache.org/r/661/diff
Testing
-------
Test-patch:
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
Unit test:
all pass
Thanks,
Daniel
> mapred.output.compress in SET statement does not work
> -----------------------------------------------------
>
> Key: PIG-1814
> URL: https://issues.apache.org/jira/browse/PIG-1814
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: PIG-1814-1.patch
>
>
> Setting output compression using "SET" in the script does not work:
> SET mapred.output.compress true;
> SET mapred.output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> We did some trick to make individual compression setting for multistore work. Instead of the above parameter, using the following works:
> SET output.compression.enabled true;
> SET output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> However, this is against intuition. We should use mapred.output.compress/mapred.output.compression.codec.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-1814) mapred.output.compress in SET
statement does not work
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai resolved PIG-1814.
-----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Patch committed to both trunk and 0.9 branch.
> mapred.output.compress in SET statement does not work
> -----------------------------------------------------
>
> Key: PIG-1814
> URL: https://issues.apache.org/jira/browse/PIG-1814
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.9.0
>
> Attachments: PIG-1814-1.patch
>
>
> Setting output compression using "SET" in the script does not work:
> SET mapred.output.compress true;
> SET mapred.output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> We did some trick to make individual compression setting for multistore work. Instead of the above parameter, using the following works:
> SET output.compression.enabled true;
> SET output.compression.codec org.apache.hadoop.io.compress.GzipCodec;
> However, this is against intuition. We should use mapred.output.compress/mapred.output.compression.codec.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira