You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Kang Xiao (Created) (JIRA)" <ji...@apache.org> on 2011/11/21 09:34:51 UTC
[jira] [Created] (HADOOP-7842) a compress tool for test
compress/decompress locally use hadoop compress codecs
a compress tool for test compress/decompress locally use hadoop compress codecs
-------------------------------------------------------------------------------
Key: HADOOP-7842
URL: https://issues.apache.org/jira/browse/HADOOP-7842
Project: Hadoop Common
Issue Type: Improvement
Components: io
Reporter: Kang Xiao
Assignee: Kang Xiao
add a compress tool for test compress/decompress locally use hadoop compress codecs.
It can be used as follows:
compress a.txt to a.lzo:
hadoop jar test.jar org.apache.hadoop.io.compress.CodecTool org.apache.hadoop.io.compress.LzoCodec -c 1024 1024 a.txt a.lzo
decompress a.lzo to stdout:
hadoop jar test.jar org.apache.hadoop.io.compress.CodecTool org.apache.hadoop.io.compress.LzoCodec -x 1024 1024 a.lzo /dev/stdout
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7842) a compress tool for test
compress/decompress locally use hadoop compress codecs
Posted by "Kang Xiao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kang Xiao updated HADOOP-7842:
------------------------------
Attachment: HADOOP-7842.patch
patch for review.
> a compress tool for test compress/decompress locally use hadoop compress codecs
> -------------------------------------------------------------------------------
>
> Key: HADOOP-7842
> URL: https://issues.apache.org/jira/browse/HADOOP-7842
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Kang Xiao
> Assignee: Kang Xiao
> Labels: compression
> Attachments: HADOOP-7842.patch
>
>
> add a compress tool for test compress/decompress locally use hadoop compress codecs.
> It can be used as follows:
> compress a.txt to a.lzo:
> hadoop jar test.jar org.apache.hadoop.io.compress.CodecTool org.apache.hadoop.io.compress.LzoCodec -c 1024 1024 a.txt a.lzo
> decompress a.lzo to stdout:
> hadoop jar test.jar org.apache.hadoop.io.compress.CodecTool org.apache.hadoop.io.compress.LzoCodec -x 1024 1024 a.lzo /dev/stdout
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7842) a compress tool for test
compress/decompress locally use hadoop compress codecs
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154091#comment-13154091 ]
Harsh J commented on HADOOP-7842:
---------------------------------
Hello Kang,
Could you explain the rationale behind this? Files are already decompressed by {{-cat}}, etc. shell utils, native programs like lzo's.
If its for testing native lib loads, or compression codec loading, we already have TestCodec for that, which I've once explained how to use at: http://search-hadoop.com/m/RqwE52Sh1GI
Given these two facts, I believe the only usefulness of adding this is to compress a file given a codec? Perhaps TestCodec may be extended to provide such a utility as well?
> a compress tool for test compress/decompress locally use hadoop compress codecs
> -------------------------------------------------------------------------------
>
> Key: HADOOP-7842
> URL: https://issues.apache.org/jira/browse/HADOOP-7842
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Kang Xiao
> Assignee: Kang Xiao
> Labels: compression
> Attachments: HADOOP-7842.patch
>
>
> add a compress tool for test compress/decompress locally use hadoop compress codecs.
> It can be used as follows:
> compress a.txt to a.lzo:
> hadoop jar test.jar org.apache.hadoop.io.compress.CodecTool org.apache.hadoop.io.compress.LzoCodec -c 1024 1024 a.txt a.lzo
> decompress a.lzo to stdout:
> hadoop jar test.jar org.apache.hadoop.io.compress.CodecTool org.apache.hadoop.io.compress.LzoCodec -x 1024 1024 a.lzo /dev/stdout
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7842) a compress tool for test
compress/decompress locally use hadoop compress codecs
Posted by "Kang Xiao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154078#comment-13154078 ]
Kang Xiao commented on HADOOP-7842:
-----------------------------------
Is it suitable to put it under the test directory, and suggestion?
> a compress tool for test compress/decompress locally use hadoop compress codecs
> -------------------------------------------------------------------------------
>
> Key: HADOOP-7842
> URL: https://issues.apache.org/jira/browse/HADOOP-7842
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Kang Xiao
> Assignee: Kang Xiao
> Labels: compression
> Attachments: HADOOP-7842.patch
>
>
> add a compress tool for test compress/decompress locally use hadoop compress codecs.
> It can be used as follows:
> compress a.txt to a.lzo:
> hadoop jar test.jar org.apache.hadoop.io.compress.CodecTool org.apache.hadoop.io.compress.LzoCodec -c 1024 1024 a.txt a.lzo
> decompress a.lzo to stdout:
> hadoop jar test.jar org.apache.hadoop.io.compress.CodecTool org.apache.hadoop.io.compress.LzoCodec -x 1024 1024 a.lzo /dev/stdout
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira