You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Thomas Graves (Created) (JIRA)" <ji...@apache.org> on 2011/09/30 00:13:45 UTC
[jira] [Created] (MAPREDUCE-3124) mapper failed with failed to load
native libs
mapper failed with failed to load native libs
---------------------------------------------
Key: MAPREDUCE-3124
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Priority: Blocker
Fix For: 0.23.0
hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
-job=* -Dmapreduce.map.output.compress=true
-Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
-Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
The error in container log:
2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
org.apache.hadoop.util.NativeCodeLoader:
java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
using builtin-java classes where applicable
Also note that the error that shows up at the application master for this is terrible:
Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126669#comment-13126669 ]
Hadoop QA commented on MAPREDUCE-3124:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12498887/MAPREDUCE-3124.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1005//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1005//console
This message is automatically generated.
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127526#comment-13127526 ]
Hudson commented on MAPREDUCE-3124:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #51 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/51/])
Merge -c 1183188 from trunk to branch-0.23 to fix MAPREDUCE-3124.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183189
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3124) mapper failed with failed to load
native libs
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-3124:
-----------------------------------
Attachment: MAPREDUCE-3124.patch
Thanks Vinod. Attaching another patch using getPlatform().
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126266#comment-13126266 ]
Hadoop QA commented on MAPREDUCE-3124:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12498840/MAPREDUCE-3124.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1000//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1000//console
This message is automatically generated.
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "John George (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George reassigned MAPREDUCE-3124:
--------------------------------------
Assignee: John George (was: Thomas Graves)
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127484#comment-13127484 ]
Hudson commented on MAPREDUCE-3124:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #39 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/39/])
Merge -c 1183188 from trunk to branch-0.23 to fix MAPREDUCE-3124.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183189
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127218#comment-13127218 ]
Hudson commented on MAPREDUCE-3124:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1158 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1158/])
MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for containers. Contributed by John George.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183188
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127236#comment-13127236 ]
Hudson commented on MAPREDUCE-3124:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1099 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1099/])
MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for containers. Contributed by John George.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183188
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127216#comment-13127216 ]
Hudson commented on MAPREDUCE-3124:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1080 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1080/])
MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for containers. Contributed by John George.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183188
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127519#comment-13127519 ]
Hudson commented on MAPREDUCE-3124:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #860 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/860/])
MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for containers. Contributed by John George.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183188
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3124) mapper failed with failed to load
native libs
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-3124:
-----------------------------------
Affects Version/s: 0.24.0
Status: Patch Available (was: Open)
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3124) mapper failed with failed to load
native libs
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3124:
-----------------------------------------------
Status: Open (was: Patch Available)
Hard coding "Linux-i386-32" won't fly. I guess you can use the {{PlatformName}} class in Hadoop common utils package, but please test and verify. Thanks.
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127500#comment-13127500 ]
Hudson commented on MAPREDUCE-3124:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #830 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/830/])
MAPREDUCE-3124. Fixed location of native libs i.e. libhadoop.so for containers. Contributed by John George.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183188
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3124) mapper failed with failed to
load native libs
Posted by "Thomas Graves (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves reassigned MAPREDUCE-3124:
----------------------------------------
Assignee: Thomas Graves
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: Thomas Graves
> Priority: Blocker
> Fix For: 0.23.0
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3124) mapper failed with failed to load
native libs
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-3124:
-----------------------------------
Status: Patch Available (was: Open)
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3124) mapper failed with failed to load
native libs
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-3124:
-----------------------------------
Attachment: MAPREDUCE-3124.patch
Uploading a patch to check if the fix is in the right direction...
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3124) mapper failed with failed to load
native libs
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3124:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this trivial change. Thanks John!
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3124) mapper failed with failed to load
native libs
Posted by "John George (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John George updated MAPREDUCE-3124:
-----------------------------------
Attachment: MAPREDUCE-3124.patch
> mapper failed with failed to load native libs
> ---------------------------------------------
>
> Key: MAPREDUCE-3124
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3124
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Thomas Graves
> Assignee: John George
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-3124.patch, MAPREDUCE-3124.patch
>
>
> hadoop jar hadoop-mapreduce-examples-*.jar sort -Dmapreduce.job.acl-view
> -job=* -Dmapreduce.map.output.compress=true
> -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec
> -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compression.type=NONE -Dmap
> reduce.output.fileoutputformat.compression.codec=org.apache.hadoop.io.compress.GzipCodec -outKey
> org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text Compression/textinput Compression/textoutput-1317315994
> This will fail with native libs not found error unless -Dmapred.child.java.opts='-Djava.library.path=${HADOOP_COMMON_HOME}/lib/native/Linux-i386-32' is added.
> The error in container log:
> 2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built native-hadoop
> library...2011-09-29 17:06:56,787 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop with
> error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path2011-09-29 17:06:56,787 DEBUG
> org.apache.hadoop.util.NativeCodeLoader:
> java.library.path=/share/gridjdk-1.6.0_21/jre/lib/i386/server:/share/gridjdk-1.6.0_21/jre/lib/i386:/share/gridjdk-1.6.0_21/jre/../lib/i386:/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317314754104_0012/container_1317314754104_0012_01_000002:/current/lib:/usr/java/packages/lib/i386:/lib:/usr/lib2011-09-29
> 17:06:56,787 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
> using builtin-java classes where applicable
> Also note that the error that shows up at the application master for this is terrible:
> Container killed by the ApplicationMaster. Container killed on request. Exit code is 137 Too Many fetch failures.Failing the attempt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira