You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2013/08/06 08:39:10 UTC
issue about try lzo in hadoop
i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run test
,it failed,why?
# sudo -u hdfs hadoop jar
/usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
com.hadoop.compression.lzo.LzoIndexer /test
13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is deprecated.
Instead, use io.native.lib.available
13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
seconds (0.05 MB/s). Index size is 0.01 KB.
[root@node1 ~]# echo $?
0
]# sudo -u hdfs hadoop jar
/usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
com.hadoop.compression.lzo.DistributedLzoIndexer /test
13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
exists)
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/Job
at
com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.Job
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 9 more
Re: issue about try lzo in hadoop
Posted by ch huang <ju...@gmail.com>.
i installed
hadoop-0.20-mapreduce-2.0.0+1357-1.cdh4.3.0.p0.21.el6.x86_64,and now ok,
question is test code use MRV1,not YARN?
On Tue, Aug 6, 2013 at 3:27 PM, Sandeep Nemuri <nh...@gmail.com> wrote:
> send me your conf file .
>
>
> On Tue, Aug 6, 2013 at 12:09 PM, ch huang <ju...@gmail.com> wrote:
>
>> i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run
>> test ,it failed,why?
>>
>> # sudo -u hdfs hadoop jar
>> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
>> com.hadoop.compression.lzo.LzoIndexer /test
>> 13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
>> hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
>> 13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is
>> deprecated. Instead, use io.native.lib.available
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
>> seconds (0.05 MB/s). Index size is 0.01 KB.
>> [root@node1 ~]# echo $?
>> 0
>> ]# sudo -u hdfs hadoop jar
>> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
>> com.hadoop.compression.lzo.DistributedLzoIndexer /test
>> 13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
>> 13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
>> hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
>> exists)
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/hadoop/mapreduce/Job
>> at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.mapreduce.Job
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
>> ... 9 more
>>
>
>
>
> --
> --Regards
> Sandeep Nemuri
>
Re: issue about try lzo in hadoop
Posted by ch huang <ju...@gmail.com>.
i installed
hadoop-0.20-mapreduce-2.0.0+1357-1.cdh4.3.0.p0.21.el6.x86_64,and now ok,
question is test code use MRV1,not YARN?
On Tue, Aug 6, 2013 at 3:27 PM, Sandeep Nemuri <nh...@gmail.com> wrote:
> send me your conf file .
>
>
> On Tue, Aug 6, 2013 at 12:09 PM, ch huang <ju...@gmail.com> wrote:
>
>> i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run
>> test ,it failed,why?
>>
>> # sudo -u hdfs hadoop jar
>> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
>> com.hadoop.compression.lzo.LzoIndexer /test
>> 13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
>> hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
>> 13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is
>> deprecated. Instead, use io.native.lib.available
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
>> seconds (0.05 MB/s). Index size is 0.01 KB.
>> [root@node1 ~]# echo $?
>> 0
>> ]# sudo -u hdfs hadoop jar
>> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
>> com.hadoop.compression.lzo.DistributedLzoIndexer /test
>> 13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
>> 13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
>> hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
>> exists)
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/hadoop/mapreduce/Job
>> at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.mapreduce.Job
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
>> ... 9 more
>>
>
>
>
> --
> --Regards
> Sandeep Nemuri
>
Re: issue about try lzo in hadoop
Posted by ch huang <ju...@gmail.com>.
i installed
hadoop-0.20-mapreduce-2.0.0+1357-1.cdh4.3.0.p0.21.el6.x86_64,and now ok,
question is test code use MRV1,not YARN?
On Tue, Aug 6, 2013 at 3:27 PM, Sandeep Nemuri <nh...@gmail.com> wrote:
> send me your conf file .
>
>
> On Tue, Aug 6, 2013 at 12:09 PM, ch huang <ju...@gmail.com> wrote:
>
>> i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run
>> test ,it failed,why?
>>
>> # sudo -u hdfs hadoop jar
>> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
>> com.hadoop.compression.lzo.LzoIndexer /test
>> 13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
>> hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
>> 13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is
>> deprecated. Instead, use io.native.lib.available
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
>> seconds (0.05 MB/s). Index size is 0.01 KB.
>> [root@node1 ~]# echo $?
>> 0
>> ]# sudo -u hdfs hadoop jar
>> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
>> com.hadoop.compression.lzo.DistributedLzoIndexer /test
>> 13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
>> 13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
>> hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
>> exists)
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/hadoop/mapreduce/Job
>> at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.mapreduce.Job
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
>> ... 9 more
>>
>
>
>
> --
> --Regards
> Sandeep Nemuri
>
Re: issue about try lzo in hadoop
Posted by ch huang <ju...@gmail.com>.
i installed
hadoop-0.20-mapreduce-2.0.0+1357-1.cdh4.3.0.p0.21.el6.x86_64,and now ok,
question is test code use MRV1,not YARN?
On Tue, Aug 6, 2013 at 3:27 PM, Sandeep Nemuri <nh...@gmail.com> wrote:
> send me your conf file .
>
>
> On Tue, Aug 6, 2013 at 12:09 PM, ch huang <ju...@gmail.com> wrote:
>
>> i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run
>> test ,it failed,why?
>>
>> # sudo -u hdfs hadoop jar
>> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
>> com.hadoop.compression.lzo.LzoIndexer /test
>> 13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
>> hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
>> 13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is
>> deprecated. Instead, use io.native.lib.available
>> 13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
>> seconds (0.05 MB/s). Index size is 0.01 KB.
>> [root@node1 ~]# echo $?
>> 0
>> ]# sudo -u hdfs hadoop jar
>> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
>> com.hadoop.compression.lzo.DistributedLzoIndexer /test
>> 13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
>> 13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
>> hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
>> exists)
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/hadoop/mapreduce/Job
>> at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.mapreduce.Job
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
>> ... 9 more
>>
>
>
>
> --
> --Regards
> Sandeep Nemuri
>
Re: issue about try lzo in hadoop
Posted by Sandeep Nemuri <nh...@gmail.com>.
send me your conf file .
On Tue, Aug 6, 2013 at 12:09 PM, ch huang <ju...@gmail.com> wrote:
> i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run
> test ,it failed,why?
>
> # sudo -u hdfs hadoop jar
> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.LzoIndexer /test
> 13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
> hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
> 13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
> seconds (0.05 MB/s). Index size is 0.01 KB.
> [root@node1 ~]# echo $?
> 0
> ]# sudo -u hdfs hadoop jar
> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.DistributedLzoIndexer /test
> 13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
> 13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
> hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
> exists)
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/Job
> at
> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.Job
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> ... 9 more
>
--
--Regards
Sandeep Nemuri
Re: issue about try lzo in hadoop
Posted by Sandeep Nemuri <nh...@gmail.com>.
send me your conf file .
On Tue, Aug 6, 2013 at 12:09 PM, ch huang <ju...@gmail.com> wrote:
> i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run
> test ,it failed,why?
>
> # sudo -u hdfs hadoop jar
> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.LzoIndexer /test
> 13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
> hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
> 13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
> seconds (0.05 MB/s). Index size is 0.01 KB.
> [root@node1 ~]# echo $?
> 0
> ]# sudo -u hdfs hadoop jar
> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.DistributedLzoIndexer /test
> 13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
> 13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
> hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
> exists)
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/Job
> at
> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.Job
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> ... 9 more
>
--
--Regards
Sandeep Nemuri
Re: issue about try lzo in hadoop
Posted by Sandeep Nemuri <nh...@gmail.com>.
send me your conf file .
On Tue, Aug 6, 2013 at 12:09 PM, ch huang <ju...@gmail.com> wrote:
> i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run
> test ,it failed,why?
>
> # sudo -u hdfs hadoop jar
> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.LzoIndexer /test
> 13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
> hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
> 13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
> seconds (0.05 MB/s). Index size is 0.01 KB.
> [root@node1 ~]# echo $?
> 0
> ]# sudo -u hdfs hadoop jar
> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.DistributedLzoIndexer /test
> 13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
> 13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
> hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
> exists)
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/Job
> at
> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.Job
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> ... 9 more
>
--
--Regards
Sandeep Nemuri
Re: issue about try lzo in hadoop
Posted by Sandeep Nemuri <nh...@gmail.com>.
send me your conf file .
On Tue, Aug 6, 2013 at 12:09 PM, ch huang <ju...@gmail.com> wrote:
> i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run
> test ,it failed,why?
>
> # sudo -u hdfs hadoop jar
> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.LzoIndexer /test
> 13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
> hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
> 13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 13/08/06 22:28:04 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.18
> seconds (0.05 MB/s). Index size is 0.01 KB.
> [root@node1 ~]# echo $?
> 0
> ]# sudo -u hdfs hadoop jar
> /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.DistributedLzoIndexer /test
> 13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
> 13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
> hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
> exists)
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/Job
> at
> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.Job
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> ... 9 more
>
--
--Regards
Sandeep Nemuri