You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Matt Davies <ma...@tynt.com> on 2011/05/25 23:08:46 UTC
Using MiniHBaseCluster in Maven Tests with LZO
Hey all,
Running into a situation where I'm trying to use MiniHBaseCluster to test
operations against a table with LZO compression enabled. I have a set of
unit tests that exercise certain capabilities, but the moment I use LZO it
fails. I know exactly that it is that it cannot find the
nativegplcompression (java.lang.UnsatisfiedLinkError: no gplcompression in
java.library.path), but I'm not seeing how to add it into the path using
Maven.
I've installed the LZO libraries for Mac as described at
http://code.google.com/a/apache-extras.org/p/hadoop-gpl-compression/wiki/FAQ?redir=1
I've tried adding the java.library.path into the pom as such:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemProperties>
<property>
<name>java.library.path</name>
<value>/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/</value>
</property>
</systemProperties>
</configuration>
</plugin>
(I've also tried all sorts of combinations of the pathing with and without
the Mac_OS_X...)
When the MiniHBaseCluster starts up Zookeeper reports that it sees the path
INFO server.ZooKeeperServer: Server
environment:java.library.path=/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/
But as it gets to the meat of the operation it fails.
java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
...
11/05/25 14:57:07 ERROR lzo.LzoCodec: Cannot load native-lzo without
native-hadoop
11/05/25 14:57:07 ERROR handler.OpenRegionHandler: Failed open of
region=Base,,1306357026963.cbb568b5b264caafca3b9e0121ccdca8.
java.io.IOException: java.lang.RuntimeException: native-lzo library not
available
...
Caused by: java.lang.RuntimeException: native-lzo library not available
I'd like to be able to put tests with my LZO enabled table. Suggestions?
Re: Using MiniHBaseCluster in Maven Tests with LZO
Posted by Matt Davies <ma...@mattdavies.net>.
I've figured it out. Turns out that the other configuration was loaded too
late. Regardless the following works properly:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<forkMode>once</forkMode>
<workingDirectory>target</workingDirectory>
<argLine>-Djava.library.path=/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/</argLine>
</configuration>
</plugin>
Thanks to post at
http://maven.40175.n5.nabble.com/Trouble-with-Java-Native-Libraries-td114063.html
On Wed, May 25, 2011 at 3:08 PM, Matt Davies <ma...@tynt.com> wrote:
> Hey all,
>
> Running into a situation where I'm trying to use MiniHBaseCluster to test
> operations against a table with LZO compression enabled. I have a set of
> unit tests that exercise certain capabilities, but the moment I use LZO it
> fails. I know exactly that it is that it cannot find the
> nativegplcompression (java.lang.UnsatisfiedLinkError: no gplcompression in
> java.library.path), but I'm not seeing how to add it into the path using
> Maven.
>
> I've installed the LZO libraries for Mac as described at
>
> http://code.google.com/a/apache-extras.org/p/hadoop-gpl-compression/wiki/FAQ?redir=1
>
> I've tried adding the java.library.path into the pom as such:
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <configuration>
> <systemProperties>
> <property>
> <name>java.library.path</name>
>
> <value>/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/</value>
> </property>
> </systemProperties>
> </configuration>
> </plugin>
>
> (I've also tried all sorts of combinations of the pathing with and without
> the Mac_OS_X...)
>
> When the MiniHBaseCluster starts up Zookeeper reports that it sees the path
> INFO server.ZooKeeperServer: Server
>
> environment:java.library.path=/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/
>
> But as it gets to the meat of the operation it fails.
>
> java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
> ...
> 11/05/25 14:57:07 ERROR lzo.LzoCodec: Cannot load native-lzo without
> native-hadoop
> 11/05/25 14:57:07 ERROR handler.OpenRegionHandler: Failed open of
> region=Base,,1306357026963.cbb568b5b264caafca3b9e0121ccdca8.
> java.io.IOException: java.lang.RuntimeException: native-lzo library not
> available
> ...
> Caused by: java.lang.RuntimeException: native-lzo library not available
>
>
> I'd like to be able to put tests with my LZO enabled table. Suggestions?
>