You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Vadim Keylis <vk...@gmail.com> on 2011/04/19 07:41:37 UTC
HBase 0.90.2 CDH3B4 -Compression algorithm 'lzo' previously failed test
Good evening. I've read the article(
https://github.com/toddlipcon/hadoop-lzo) and configured the way it
describes, but I still get the same error.
2011-04-18 15:09:11,635 ERROR
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
of
region=user,10L\xA2\x13\xDB\xDB\xDB\xDB\xDA\xDA\xB0\x05Z\xCCw!\xCC\x93\xB0!\xCC\x93\x93\xB0!\xCCw!\xCCw!\xCCw
,1303164462772.dbbb665cfc461075670f06f4dfe6b632.
java.io.IOException: Compression algorithm 'lzo' previously failed test.
at
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2555)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2544)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2532)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:262)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:94)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:151)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
I check out hadoop-lzo from the link you provided and rebuild the library
based on the instruction in FAQ.
Here my configuration:
hadoop-env.sh:
export
JAVA_LIBRARY_PATH="/home/hbase/hadoop-lzo-0.4.10/lib/native/Linux-amd64-64/"
# Extra Java CLASSPATH elements. Optional.
export
HADOOP_CLASSPATH="/home/hbase/hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar"
core-site.xml
<property>
<name>io.compression.codecs</name>
<value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.BZip2Codec</value>
</property>
<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
hbase-env.sh:
export HBASE_CLASSPATH="/home/hbase/
hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar:$HBASE_CLASSPATH"
What am I doing wrong
Re: HBase 0.90.2 CDH3B4 -Compression algorithm 'lzo' previously
failed test
Posted by Stack <st...@duboce.net>.
What was the issue (so the rest of us can learn from your experience?).
Thanks Vadim,
St.Ack
On Tue, Apr 19, 2011 at 10:20 AM, Vadim Keylis <vk...@gmail.com> wrote:
> Thanks so much. Figure the problem that caused lzo not to work.
>
> Thanks again.
>
> On Tue, Apr 19, 2011 at 9:50 AM, Vadim Keylis <vk...@gmail.com> wrote:
>>
>> Good morning. No it seems test was successful. Native libraries located in
>> /home/
>> hbase/hadoop-lzo-0.4.10/lib/native/Linux-amd64-64/ that were compiled
>> after built project.
>>
>> [hbase@dhbasetest01 shell]$ ../hbase/bin/hbase
>> org.apache.hadoop.hbase.util.CompressionTest
>> hdfs://dhbasetest01.tag-dev.com:9000/hbase lzo
>> 11/04/19 09:47:39 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 11/04/19 09:47:40 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 4a3bd479610257d27e4ece33ce3f772ea6830d6d]
>> 11/04/19 09:47:40 INFO compress.CodecPool: Got brand-new compressor
>>
>> On Mon, Apr 18, 2011 at 10:47 PM, Stack <st...@duboce.net> wrote:
>>>
>>> It looks like its still not installed correctly; HBase when it runs
>>> can't find it (You restarted between installs? What does the
>>> CompressionTool say when you run it? See
>>> http://hbase.apache.org/book.html#compression The same thing?) Where
>>> did you put the native libs?
>>>
>>> St.Ack
>>>
>>>
>>>
>>> On Mon, Apr 18, 2011 at 10:41 PM, Vadim Keylis <vk...@gmail.com>
>>> wrote:
>>> > Good evening. I've read the article(
>>> > https://github.com/toddlipcon/hadoop-lzo) and configured the way it
>>> > describes, but I still get the same error.
>>> >
>>> > 2011-04-18 15:09:11,635 ERROR
>>> > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
>>> > open
>>> > of
>>> >
>>> > region=user,10L\xA2\x13\xDB\xDB\xDB\xDB\xDA\xDA\xB0\x05Z\xCCw!\xCC\x93\xB0!\xCC\x93\x93\xB0!\xCCw!\xCCw!\xCCw
>>> > ,1303164462772.dbbb665cfc461075670f06f4dfe6b632.
>>> >
>>> > java.io.IOException: Compression algorithm 'lzo' previously failed
>>> > test.
>>> > at
>>> >
>>> > org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:77)
>>> > at
>>> >
>>> > org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2555)
>>> > at
>>> >
>>> > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2544)
>>> > at
>>> >
>>> > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2532)
>>> > at
>>> >
>>> > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:262)
>>> > at
>>> >
>>> > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:94)
>>> > at
>>> >
>>> > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:151)
>>> > at
>>> >
>>> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> > at
>>> >
>>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> > at java.lang.Thread.run(Thread.java:619)
>>> >
>>> >
>>> > I check out hadoop-lzo from the link you provided and rebuild the
>>> > library
>>> > based on the instruction in FAQ.
>>> >
>>> > Here my configuration:
>>> >
>>> > hadoop-env.sh:
>>> >
>>> > export
>>> >
>>> > JAVA_LIBRARY_PATH="/home/hbase/hadoop-lzo-0.4.10/lib/native/Linux-amd64-64/"
>>> > # Extra Java CLASSPATH elements. Optional.
>>> > export
>>> > HADOOP_CLASSPATH="/home/hbase/hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar"
>>> >
>>> >
>>> > core-site.xml
>>> >
>>> > <property>
>>> > <name>io.compression.codecs</name>
>>> >
>>> >
>>> > <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.BZip2Codec</value>
>>> > </property>
>>> > <property>
>>> > <name>io.compression.codec.lzo.class</name>
>>> > <value>com.hadoop.compression.lzo.LzoCodec</value>
>>> > </property>
>>> >
>>> > hbase-env.sh:
>>> >
>>> > export HBASE_CLASSPATH="/home/hbase/
>>> > hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar:$HBASE_CLASSPATH"
>>> >
>>> > What am I doing wrong
>>> >
>>> >
>>
>
>
Re: HBase 0.90.2 CDH3B4 -Compression algorithm 'lzo' previously
failed test
Posted by Vadim Keylis <vk...@gmail.com>.
Thanks so much. Figure the problem that caused lzo not to work.
Thanks again.
On Tue, Apr 19, 2011 at 9:50 AM, Vadim Keylis <vk...@gmail.com> wrote:
> Good morning. No it seems test was successful. Native libraries located in
> /home/
> hbase/hadoop-lzo-0.4.10/lib/native/Linux-amd64-64/ that were compiled after
> built project.
>
> [hbase@dhbasetest01 shell]$ ../hbase/bin/hbase
> org.apache.hadoop.hbase.util.CompressionTest hdfs://
> dhbasetest01.tag-dev.com:9000/hbase lzo
> 11/04/19 09:47:39 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 11/04/19 09:47:40 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 4a3bd479610257d27e4ece33ce3f772ea6830d6d]
> 11/04/19 09:47:40 INFO compress.CodecPool: Got brand-new compressor
>
>
> On Mon, Apr 18, 2011 at 10:47 PM, Stack <st...@duboce.net> wrote:
>
>> It looks like its still not installed correctly; HBase when it runs
>> can't find it (You restarted between installs? What does the
>> CompressionTool say when you run it? See
>> http://hbase.apache.org/book.html#compression The same thing?) Where
>> did you put the native libs?
>>
>> St.Ack
>>
>>
>>
>> On Mon, Apr 18, 2011 at 10:41 PM, Vadim Keylis <vk...@gmail.com>
>> wrote:
>> > Good evening. I've read the article(
>> > https://github.com/toddlipcon/hadoop-lzo) and configured the way it
>> > describes, but I still get the same error.
>> >
>> > 2011-04-18 15:09:11,635 ERROR
>> > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
>> open
>> > of
>> >
>> region=user,10L\xA2\x13\xDB\xDB\xDB\xDB\xDA\xDA\xB0\x05Z\xCCw!\xCC\x93\xB0!\xCC\x93\x93\xB0!\xCCw!\xCCw!\xCCw
>> > ,1303164462772.dbbb665cfc461075670f06f4dfe6b632.
>> >
>> > java.io.IOException: Compression algorithm 'lzo' previously failed test.
>> > at
>> >
>> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:77)
>> > at
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2555)
>> > at
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2544)
>> > at
>> >
>> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2532)
>> > at
>> >
>> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:262)
>> > at
>> >
>> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:94)
>> > at
>> > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:151)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> > at java.lang.Thread.run(Thread.java:619)
>> >
>> >
>> > I check out hadoop-lzo from the link you provided and rebuild the
>> library
>> > based on the instruction in FAQ.
>> >
>> > Here my configuration:
>> >
>> > hadoop-env.sh:
>> >
>> > export
>> >
>> JAVA_LIBRARY_PATH="/home/hbase/hadoop-lzo-0.4.10/lib/native/Linux-amd64-64/"
>> > # Extra Java CLASSPATH elements. Optional.
>> > export
>> > HADOOP_CLASSPATH="/home/hbase/hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar"
>> >
>> >
>> > core-site.xml
>> >
>> > <property>
>> > <name>io.compression.codecs</name>
>> >
>> >
>> <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.BZip2Codec</value>
>> > </property>
>> > <property>
>> > <name>io.compression.codec.lzo.class</name>
>> > <value>com.hadoop.compression.lzo.LzoCodec</value>
>> > </property>
>> >
>> > hbase-env.sh:
>> >
>> > export HBASE_CLASSPATH="/home/hbase/
>> > hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar:$HBASE_CLASSPATH"
>> >
>> > What am I doing wrong
>> >
>> >
>>
>
>
Re: HBase 0.90.2 CDH3B4 -Compression algorithm 'lzo' previously
failed test
Posted by Vadim Keylis <vk...@gmail.com>.
Good morning. No it seems test was successful. Native libraries located in
/home/
hbase/hadoop-lzo-0.4.10/lib/native/Linux-amd64-64/ that were compiled after
built project.
[hbase@dhbasetest01 shell]$ ../hbase/bin/hbase
org.apache.hadoop.hbase.util.CompressionTest hdfs://
dhbasetest01.tag-dev.com:9000/hbase lzo
11/04/19 09:47:39 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
11/04/19 09:47:40 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 4a3bd479610257d27e4ece33ce3f772ea6830d6d]
11/04/19 09:47:40 INFO compress.CodecPool: Got brand-new compressor
On Mon, Apr 18, 2011 at 10:47 PM, Stack <st...@duboce.net> wrote:
> It looks like its still not installed correctly; HBase when it runs
> can't find it (You restarted between installs? What does the
> CompressionTool say when you run it? See
> http://hbase.apache.org/book.html#compression The same thing?) Where
> did you put the native libs?
>
> St.Ack
>
>
>
> On Mon, Apr 18, 2011 at 10:41 PM, Vadim Keylis <vk...@gmail.com>
> wrote:
> > Good evening. I've read the article(
> > https://github.com/toddlipcon/hadoop-lzo) and configured the way it
> > describes, but I still get the same error.
> >
> > 2011-04-18 15:09:11,635 ERROR
> > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
> open
> > of
> >
> region=user,10L\xA2\x13\xDB\xDB\xDB\xDB\xDA\xDA\xB0\x05Z\xCCw!\xCC\x93\xB0!\xCC\x93\x93\xB0!\xCCw!\xCCw!\xCCw
> > ,1303164462772.dbbb665cfc461075670f06f4dfe6b632.
> >
> > java.io.IOException: Compression algorithm 'lzo' previously failed test.
> > at
> >
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:77)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2555)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2544)
> > at
> >
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2532)
> > at
> >
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:262)
> > at
> >
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:94)
> > at
> > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:151)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > at java.lang.Thread.run(Thread.java:619)
> >
> >
> > I check out hadoop-lzo from the link you provided and rebuild the library
> > based on the instruction in FAQ.
> >
> > Here my configuration:
> >
> > hadoop-env.sh:
> >
> > export
> >
> JAVA_LIBRARY_PATH="/home/hbase/hadoop-lzo-0.4.10/lib/native/Linux-amd64-64/"
> > # Extra Java CLASSPATH elements. Optional.
> > export
> > HADOOP_CLASSPATH="/home/hbase/hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar"
> >
> >
> > core-site.xml
> >
> > <property>
> > <name>io.compression.codecs</name>
> >
> >
> <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.BZip2Codec</value>
> > </property>
> > <property>
> > <name>io.compression.codec.lzo.class</name>
> > <value>com.hadoop.compression.lzo.LzoCodec</value>
> > </property>
> >
> > hbase-env.sh:
> >
> > export HBASE_CLASSPATH="/home/hbase/
> > hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar:$HBASE_CLASSPATH"
> >
> > What am I doing wrong
> >
> >
>
Re: HBase 0.90.2 CDH3B4 -Compression algorithm 'lzo' previously
failed test
Posted by Stack <st...@duboce.net>.
It looks like its still not installed correctly; HBase when it runs
can't find it (You restarted between installs? What does the
CompressionTool say when you run it? See
http://hbase.apache.org/book.html#compression The same thing?) Where
did you put the native libs?
St.Ack
On Mon, Apr 18, 2011 at 10:41 PM, Vadim Keylis <vk...@gmail.com> wrote:
> Good evening. I've read the article(
> https://github.com/toddlipcon/hadoop-lzo) and configured the way it
> describes, but I still get the same error.
>
> 2011-04-18 15:09:11,635 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open
> of
> region=user,10L\xA2\x13\xDB\xDB\xDB\xDB\xDA\xDA\xB0\x05Z\xCCw!\xCC\x93\xB0!\xCC\x93\x93\xB0!\xCCw!\xCCw!\xCCw
> ,1303164462772.dbbb665cfc461075670f06f4dfe6b632.
>
> java.io.IOException: Compression algorithm 'lzo' previously failed test.
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:77)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2555)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2544)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2532)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:262)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:94)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:151)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
>
>
> I check out hadoop-lzo from the link you provided and rebuild the library
> based on the instruction in FAQ.
>
> Here my configuration:
>
> hadoop-env.sh:
>
> export
> JAVA_LIBRARY_PATH="/home/hbase/hadoop-lzo-0.4.10/lib/native/Linux-amd64-64/"
> # Extra Java CLASSPATH elements. Optional.
> export
> HADOOP_CLASSPATH="/home/hbase/hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar"
>
>
> core-site.xml
>
> <property>
> <name>io.compression.codecs</name>
>
> <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.BZip2Codec</value>
> </property>
> <property>
> <name>io.compression.codec.lzo.class</name>
> <value>com.hadoop.compression.lzo.LzoCodec</value>
> </property>
>
> hbase-env.sh:
>
> export HBASE_CLASSPATH="/home/hbase/
> hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar:$HBASE_CLASSPATH"
>
> What am I doing wrong
>
>