You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Hitesh Shah (JIRA)" <ji...@apache.org> on 2014/03/18 22:16:51 UTC
[jira] [Assigned] (TEZ-944) Tez Job gets "Could not load native gpl
library" Error
[ https://issues.apache.org/jira/browse/TEZ-944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hitesh Shah reassigned TEZ-944:
-------------------------------
Assignee: Hitesh Shah
> Tez Job gets "Could not load native gpl library" Error
> ------------------------------------------------------
>
> Key: TEZ-944
> URL: https://issues.apache.org/jira/browse/TEZ-944
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Yesha Vora
> Assignee: Hitesh Shah
>
> Tez jobs with session are getting 'Could not load native gpl library' Error.
> {noformat}
> 2014-03-18 00:59:28,746 ERROR [InputInitializer [initialmap] #0] com.hadoop.compression.lzo.GPLNativeCodeLoader: Could not load native gpl library
> java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
> at java.lang.Runtime.loadLibrary0(Runtime.java:823)
> at java.lang.System.loadLibrary(System.java:1028)
> at com.hadoop.compression.lzo.GPLNativeCodeLoader.<clinit>(GPLNativeCodeLoader.java:32)
> at com.hadoop.compression.lzo.LzoCodec.<clinit>(LzoCodec.java:71)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1834)
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1799)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
> at org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:58)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:352)
> at org.apache.hadoop.mapreduce.split.TezGroupedSplitsInputFormat.getSplits(TezGroupedSplitsInputFormat.java:102)
> at org.apache.tez.mapreduce.hadoop.MRHelpers.generateNewSplits(MRHelpers.java:184)
> at org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:115)
> at org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable$1.run(RootInputInitializerRunner.java:145)
> at org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable$1.run(RootInputInitializerRunner.java:138)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable.call(RootInputInitializerRunner.java:138)
> at org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable.call(RootInputInitializerRunner.java:109)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2014-03-18 00:59:28,750 ERROR [InputInitializer [initialmap] #0] com.hadoop.compression.lzo.LzoCodec: Cannot load native-lzo without native-hadoop
> {noformat}
> Mapreduce has mapreduce.admin.user.env property set in mapred-site.xml
> {noformat}
> <property>
> <name>mapreduce.admin.user.env</name>
> <value>LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native:$HADOOP_COMMON_HOME/lib/native/`$JAVA_HOME/bin/java -d32 -version &> /dev/null;if [ $? -eq 0 ]; then echo Linux-i386-32; else echo Linux-amd64-64;fi`</value>
> </property>
> {noformat}
> Tez do not have any property to manage this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)