You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Matt Davies <ma...@tynt.com> on 2011/05/25 23:08:46 UTC

Using MiniHBaseCluster in Maven Tests with LZO

Hey all,

Running into a situation where I'm trying to use MiniHBaseCluster to test
operations against a table with LZO compression enabled.  I have a set of
unit tests that exercise certain capabilities, but the moment I use LZO it
fails.  I know exactly that it is that it cannot find the
nativegplcompression (java.lang.UnsatisfiedLinkError: no gplcompression in
java.library.path), but I'm not seeing how to add it into the path using
Maven.

I've installed the LZO libraries for Mac as described at
http://code.google.com/a/apache-extras.org/p/hadoop-gpl-compression/wiki/FAQ?redir=1

I've tried adding the java.library.path into the pom as such:
<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <systemProperties>
            <property>
              <name>java.library.path</name>

<value>/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/</value>
            </property>
          </systemProperties>
        </configuration>
      </plugin>

(I've also tried all sorts of combinations of the pathing with and without
the Mac_OS_X...)

When the MiniHBaseCluster starts up Zookeeper reports that it sees the path
INFO server.ZooKeeperServer: Server
environment:java.library.path=/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/

But as it gets to the meat of the operation it fails.

java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
...
11/05/25 14:57:07 ERROR lzo.LzoCodec: Cannot load native-lzo without
native-hadoop
11/05/25 14:57:07 ERROR handler.OpenRegionHandler: Failed open of
region=Base,,1306357026963.cbb568b5b264caafca3b9e0121ccdca8.
java.io.IOException: java.lang.RuntimeException: native-lzo library not
available
...
Caused by: java.lang.RuntimeException: native-lzo library not available


I'd like to be able to put tests with my LZO enabled table.  Suggestions?

Re: Using MiniHBaseCluster in Maven Tests with LZO

Posted by Matt Davies <ma...@mattdavies.net>.
I've figured it out. Turns out that the other configuration was loaded too
late. Regardless the following works properly:

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
                <forkMode>once</forkMode>
                <workingDirectory>target</workingDirectory>

<argLine>-Djava.library.path=/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/</argLine>
        </configuration>
      </plugin>

Thanks to post at
http://maven.40175.n5.nabble.com/Trouble-with-Java-Native-Libraries-td114063.html





On Wed, May 25, 2011 at 3:08 PM, Matt Davies <ma...@tynt.com> wrote:

> Hey all,
>
> Running into a situation where I'm trying to use MiniHBaseCluster to test
> operations against a table with LZO compression enabled.  I have a set of
> unit tests that exercise certain capabilities, but the moment I use LZO it
> fails.  I know exactly that it is that it cannot find the
> nativegplcompression (java.lang.UnsatisfiedLinkError: no gplcompression in
> java.library.path), but I'm not seeing how to add it into the path using
> Maven.
>
> I've installed the LZO libraries for Mac as described at
>
> http://code.google.com/a/apache-extras.org/p/hadoop-gpl-compression/wiki/FAQ?redir=1
>
> I've tried adding the java.library.path into the pom as such:
> <plugin>
>        <groupId>org.apache.maven.plugins</groupId>
>        <artifactId>maven-surefire-plugin</artifactId>
>        <configuration>
>          <systemProperties>
>            <property>
>              <name>java.library.path</name>
>
> <value>/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/</value>
>            </property>
>          </systemProperties>
>        </configuration>
>      </plugin>
>
> (I've also tried all sorts of combinations of the pathing with and without
> the Mac_OS_X...)
>
> When the MiniHBaseCluster starts up Zookeeper reports that it sees the path
> INFO server.ZooKeeperServer: Server
>
> environment:java.library.path=/Users/tynt/apps/hadoop/lib/native/Mac_OS_X-x86_64-64/
>
> But as it gets to the meat of the operation it fails.
>
> java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
> ...
> 11/05/25 14:57:07 ERROR lzo.LzoCodec: Cannot load native-lzo without
> native-hadoop
> 11/05/25 14:57:07 ERROR handler.OpenRegionHandler: Failed open of
> region=Base,,1306357026963.cbb568b5b264caafca3b9e0121ccdca8.
> java.io.IOException: java.lang.RuntimeException: native-lzo library not
> available
> ...
> Caused by: java.lang.RuntimeException: native-lzo library not available
>
>
> I'd like to be able to put tests with my LZO enabled table.  Suggestions?
>