You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org> on 2011/12/06 23:58:40 UTC
[jira] [Commented] (MAPREDUCE-3515) hadoop 0.23: native compression
libraries not being loaded
[ https://issues.apache.org/jira/browse/MAPREDUCE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163952#comment-13163952 ]
Alejandro Abdelnur commented on MAPREDUCE-3515:
-----------------------------------------------
Wing Yew, would you please verify if the build of Hadoop you were testing includes MAPREDUCE-3500? MAPREDUCE-3500 should have fixed that problem.
> hadoop 0.23: native compression libraries not being loaded
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3515
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3515
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Wing Yew Poon
>
> I installed the hadoop package from the Bigtop hadoop 0.23 branch. Among other files, the package installs
> /usr/lib/hadoop/lib/native
> /usr/lib/hadoop/lib/native/libhadoop.a
> /usr/lib/hadoop/lib/native/libhadoop.so.1
> /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0
> /usr/lib/hadoop/lib/native/libhdfs.a
> I ran a simple job using compression:
> hadoop jar /usr/lib/hadoop/hadoop-mapreduce-examples.jar wordcount -D mapreduce.output.fileoutputformat.compress=true -D mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec examples/text wordcount-gz
> I see
> 11/12/06 13:42:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> 11/12/06 13:42:06 WARN snappy.LoadSnappy: Snappy native library not loaded
> and at the end, I see
> -rw-r--r-- 1 root supergroup 0 2011-12-06 13:42 wordcount-gz/_SUCCESS
> -rw-r--r-- 1 root supergroup 46228 2011-12-06 13:42 wordcount-gz/part-r-00000.gz
> so the output is compressed, but from the log message, I assume that the native library is not being loaded and that the java gzip is being used.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira