You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "elsif (JIRA)" <ji...@apache.org> on 2009/09/30 17:07:23 UTC
[jira] Created: (HBASE-1875) Compression test utility
Compression test utility
------------------------
Key: HBASE-1875
URL: https://issues.apache.org/jira/browse/HBASE-1875
Project: Hadoop HBase
Issue Type: New Feature
Components: util
Affects Versions: 0.20.0
Reporter: elsif
Priority: Trivial
Fix For: 0.20.1
Attached is a stand alone utility to test compression compatibility
before starting hbase:
# ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
Usage: HFileCompressionTest hdfs compression
compression must be one of [ none, gz, lzo ]
# ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
hdfs://localhost:8020/ lzo
09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library
09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
OK
The utility creates a temporary HFile named '.hfile-comp-test' at the
specified HDFS path and outputs either 'OK' or 'FAILED'.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1875) Compression test utility
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1875:
-------------------------
Status: Patch Available (was: Open)
> Compression test utility
> ------------------------
>
> Key: HBASE-1875
> URL: https://issues.apache.org/jira/browse/HBASE-1875
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.20.0
> Reporter: elsif
> Priority: Trivial
> Fix For: 0.20.1
>
> Attachments: HFileCompressionTest.java
>
>
> Attached is a stand alone utility to test compression compatibility
> before starting hbase:
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> Usage: HFileCompressionTest hdfs compression
> compression must be one of [ none, gz, lzo ]
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> hdfs://localhost:8020/ lzo
> 09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library
> 09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
> OK
> The utility creates a temporary HFile named '.hfile-comp-test' at the
> specified HDFS path and outputs either 'OK' or 'FAILED'.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1875) Compression test utility
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1875:
-------------------------
Attachment: 1875-v2.patch
I added a license, formatted it to adhere to rough rules used elsewhere in the codebase, renamed it CompressionTest, and added it to o.a.h.h.util. Hope you don't mind elsif.
> Compression test utility
> ------------------------
>
> Key: HBASE-1875
> URL: https://issues.apache.org/jira/browse/HBASE-1875
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.20.0
> Reporter: elsif
> Priority: Trivial
> Fix For: 0.20.1
>
> Attachments: 1875-v2.patch, HFileCompressionTest.java
>
>
> Attached is a stand alone utility to test compression compatibility
> before starting hbase:
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> Usage: HFileCompressionTest hdfs compression
> compression must be one of [ none, gz, lzo ]
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> hdfs://localhost:8020/ lzo
> 09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library
> 09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
> OK
> The utility creates a temporary HFile named '.hfile-comp-test' at the
> specified HDFS path and outputs either 'OK' or 'FAILED'.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1875) Compression test utility
Posted by "elsif (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
elsif updated HBASE-1875:
--------------------------
Attachment: HFileCompressionTest.java
> Compression test utility
> ------------------------
>
> Key: HBASE-1875
> URL: https://issues.apache.org/jira/browse/HBASE-1875
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.20.0
> Reporter: elsif
> Priority: Trivial
> Fix For: 0.20.1
>
> Attachments: HFileCompressionTest.java
>
>
> Attached is a stand alone utility to test compression compatibility
> before starting hbase:
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> Usage: HFileCompressionTest hdfs compression
> compression must be one of [ none, gz, lzo ]
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> hdfs://localhost:8020/ lzo
> 09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library
> 09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
> OK
> The utility creates a temporary HFile named '.hfile-comp-test' at the
> specified HDFS path and outputs either 'OK' or 'FAILED'.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1875) Compression test utility
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1875:
-------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I also made it so we exit with code 0 if successfully wrote compressed file and != 0 if anything else. Updated wiki compression page to reference this new tool.
Thanks for the patch elsif. Applied branch and trunk.
> Compression test utility
> ------------------------
>
> Key: HBASE-1875
> URL: https://issues.apache.org/jira/browse/HBASE-1875
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.20.0
> Reporter: elsif
> Priority: Trivial
> Fix For: 0.20.1, 0.21.0
>
> Attachments: 1875-v2.patch, HFileCompressionTest.java
>
>
> Attached is a stand alone utility to test compression compatibility
> before starting hbase:
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> Usage: HFileCompressionTest hdfs compression
> compression must be one of [ none, gz, lzo ]
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> hdfs://localhost:8020/ lzo
> 09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library
> 09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
> OK
> The utility creates a temporary HFile named '.hfile-comp-test' at the
> specified HDFS path and outputs either 'OK' or 'FAILED'.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1875) Compression test utility
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760940#action_12760940 ]
stack commented on HBASE-1875:
------------------------------
Comment from Stephen Mulcahy:
{code}
Hi,
Thanks for this. I found it very useful this morning. Can we get this included in the standard distribution?
For those not so familiar with building and running code against hbase, the following detailed steps may be useful,
1. Download compression test util from
http://mail-archives.apache.org/mod_mbox/hadoop-hbase-user/200909.mbox/%3C4AC272
AE.3080105@gmail.com%3E
2. Compile
(change .jar's below to match your hadoop and hbase versions)
javac -cp hadoop/hadoop-0.20.1-core.jar:hbase/hbase-0.20.0.jar HFileCompressionTest.java
3. "Install"
mkdir -p build/org/apache/hadoop/hbase/test
mv HFileCompressionTest.class build/org/apache/hadoop/hbase/test
4. Run
export HBASE_CLASSPATH=build
hbase/bin/hbase org.apache.hadoop.hbase.test.HFileCompressionTest hdfs://hbase.example.org:9000/ lzo
-stephen
{code}
> Compression test utility
> ------------------------
>
> Key: HBASE-1875
> URL: https://issues.apache.org/jira/browse/HBASE-1875
> Project: Hadoop HBase
> Issue Type: New Feature
> Components: util
> Affects Versions: 0.20.0
> Reporter: elsif
> Priority: Trivial
> Fix For: 0.20.1
>
> Attachments: HFileCompressionTest.java
>
>
> Attached is a stand alone utility to test compression compatibility
> before starting hbase:
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> Usage: HFileCompressionTest hdfs compression
> compression must be one of [ none, gz, lzo ]
> # ./hbase org.apache.hadoop.hbase.test.HFileCompressionTest
> hdfs://localhost:8020/ lzo
> 09/09/29 13:39:55 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 09/09/29 13:39:55 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library
> 09/09/29 13:39:55 INFO compress.CodecPool: Got brand-new compressor
> OK
> The utility creates a temporary HFile named '.hfile-comp-test' at the
> specified HDFS path and outputs either 'OK' or 'FAILED'.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.