You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by jamal sasha <ja...@gmail.com> on 2013/08/27 17:47:38 UTC
Jar issue
Hi,
For one of my map reduce code I want to use a different version of slf4j
jar (1.6.4)
But I guess hadoop has a different version of jar in hadoop
classpath lib/slf4j-log4j12-1.4.3.jar
And when I am trying to run my code, I am gettign this error:
Exception in thread "main" java.lang.NoSuchMethodError:
org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
(SLF4JLocationAwareLog.java:159)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Similar issue here:
http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
What should I do?
Thanks
RE: Jar issue
Posted by java8964 java8964 <ja...@hotmail.com>.
I am not sure the original suggestion will work for your case.
My understanding is the you want to use some API, only exists in slf4j versiobn 1.6.4, but this library with different version already existed in your hadoop environment, which is quite possible.
To change the maven build of the application maybe not work due to:
1) If some API of 1.6.4 being used in the application code, then it must be shipped with your code into hadoop cluster.2) What you are looking for maybe is this parameter "mapreducer.user.classpath.first", you can google it, which allow the user's class files loaded before the Hadoop's in mapper/reducer tasks.3) Keep in mind that even your code should be fine now, but if the library jar you submit MAYBE not backward compatibility, then it may cause problem in the Mapper/Reducer tasks for Hadoop code, as now the slf4j 1.6.4 version of class code being loaded into JVM. But this case is very rare, as a lot of you time you will submit a later version of jar than what being contained in hadoop, and most of them are backward compatibility.4) Maybe Hadoop can use different classloader to load the user's jar file, like OLD J2EE container did, to solve this kind of problem in the future.Yong
Date: Tue, 27 Aug 2013 09:05:51 -0700
Subject: Re: Jar issue
From: jamalshasha@gmail.com
To: user@hadoop.apache.org
I am right now using libjars option.How do i do what you suggested using that route?
On Tue, Aug 27, 2013 at 8:51 AM, Shahab Yunus <sh...@gmail.com> wrote:
One idea is, you can use the exclusion property of maven (provided you are using that to build your application) while including hadoop dependencies and exclude sl4j that is coming within hadoop and then include your own sl4j as a separate dependency. Something like this:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.92.1-cdh4.1.2</version>
<exclusions>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
Regards,Shahab
RE: Jar issue
Posted by java8964 java8964 <ja...@hotmail.com>.
I am not sure the original suggestion will work for your case.
My understanding is the you want to use some API, only exists in slf4j versiobn 1.6.4, but this library with different version already existed in your hadoop environment, which is quite possible.
To change the maven build of the application maybe not work due to:
1) If some API of 1.6.4 being used in the application code, then it must be shipped with your code into hadoop cluster.2) What you are looking for maybe is this parameter "mapreducer.user.classpath.first", you can google it, which allow the user's class files loaded before the Hadoop's in mapper/reducer tasks.3) Keep in mind that even your code should be fine now, but if the library jar you submit MAYBE not backward compatibility, then it may cause problem in the Mapper/Reducer tasks for Hadoop code, as now the slf4j 1.6.4 version of class code being loaded into JVM. But this case is very rare, as a lot of you time you will submit a later version of jar than what being contained in hadoop, and most of them are backward compatibility.4) Maybe Hadoop can use different classloader to load the user's jar file, like OLD J2EE container did, to solve this kind of problem in the future.Yong
Date: Tue, 27 Aug 2013 09:05:51 -0700
Subject: Re: Jar issue
From: jamalshasha@gmail.com
To: user@hadoop.apache.org
I am right now using libjars option.How do i do what you suggested using that route?
On Tue, Aug 27, 2013 at 8:51 AM, Shahab Yunus <sh...@gmail.com> wrote:
One idea is, you can use the exclusion property of maven (provided you are using that to build your application) while including hadoop dependencies and exclude sl4j that is coming within hadoop and then include your own sl4j as a separate dependency. Something like this:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.92.1-cdh4.1.2</version>
<exclusions>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
Regards,Shahab
RE: Jar issue
Posted by java8964 java8964 <ja...@hotmail.com>.
I am not sure the original suggestion will work for your case.
My understanding is the you want to use some API, only exists in slf4j versiobn 1.6.4, but this library with different version already existed in your hadoop environment, which is quite possible.
To change the maven build of the application maybe not work due to:
1) If some API of 1.6.4 being used in the application code, then it must be shipped with your code into hadoop cluster.2) What you are looking for maybe is this parameter "mapreducer.user.classpath.first", you can google it, which allow the user's class files loaded before the Hadoop's in mapper/reducer tasks.3) Keep in mind that even your code should be fine now, but if the library jar you submit MAYBE not backward compatibility, then it may cause problem in the Mapper/Reducer tasks for Hadoop code, as now the slf4j 1.6.4 version of class code being loaded into JVM. But this case is very rare, as a lot of you time you will submit a later version of jar than what being contained in hadoop, and most of them are backward compatibility.4) Maybe Hadoop can use different classloader to load the user's jar file, like OLD J2EE container did, to solve this kind of problem in the future.Yong
Date: Tue, 27 Aug 2013 09:05:51 -0700
Subject: Re: Jar issue
From: jamalshasha@gmail.com
To: user@hadoop.apache.org
I am right now using libjars option.How do i do what you suggested using that route?
On Tue, Aug 27, 2013 at 8:51 AM, Shahab Yunus <sh...@gmail.com> wrote:
One idea is, you can use the exclusion property of maven (provided you are using that to build your application) while including hadoop dependencies and exclude sl4j that is coming within hadoop and then include your own sl4j as a separate dependency. Something like this:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.92.1-cdh4.1.2</version>
<exclusions>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
Regards,Shahab
RE: Jar issue
Posted by java8964 java8964 <ja...@hotmail.com>.
I am not sure the original suggestion will work for your case.
My understanding is the you want to use some API, only exists in slf4j versiobn 1.6.4, but this library with different version already existed in your hadoop environment, which is quite possible.
To change the maven build of the application maybe not work due to:
1) If some API of 1.6.4 being used in the application code, then it must be shipped with your code into hadoop cluster.2) What you are looking for maybe is this parameter "mapreducer.user.classpath.first", you can google it, which allow the user's class files loaded before the Hadoop's in mapper/reducer tasks.3) Keep in mind that even your code should be fine now, but if the library jar you submit MAYBE not backward compatibility, then it may cause problem in the Mapper/Reducer tasks for Hadoop code, as now the slf4j 1.6.4 version of class code being loaded into JVM. But this case is very rare, as a lot of you time you will submit a later version of jar than what being contained in hadoop, and most of them are backward compatibility.4) Maybe Hadoop can use different classloader to load the user's jar file, like OLD J2EE container did, to solve this kind of problem in the future.Yong
Date: Tue, 27 Aug 2013 09:05:51 -0700
Subject: Re: Jar issue
From: jamalshasha@gmail.com
To: user@hadoop.apache.org
I am right now using libjars option.How do i do what you suggested using that route?
On Tue, Aug 27, 2013 at 8:51 AM, Shahab Yunus <sh...@gmail.com> wrote:
One idea is, you can use the exclusion property of maven (provided you are using that to build your application) while including hadoop dependencies and exclude sl4j that is coming within hadoop and then include your own sl4j as a separate dependency. Something like this:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.92.1-cdh4.1.2</version>
<exclusions>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
Regards,Shahab
Re: Jar issue
Posted by jamal sasha <ja...@gmail.com>.
I am right now using libjars option.
How do i do what you suggested using that route?
On Tue, Aug 27, 2013 at 8:51 AM, Shahab Yunus <sh...@gmail.com>wrote:
> One idea is, you can use the exclusion property of maven (provided you are
> using that to build your application) while including hadoop dependencies
> and exclude sl4j that is coming within hadoop and then include your own
> sl4j as a separate dependency. Something like this:
>
> <dependency>
>
> <groupId>org.apache.hbase</groupId>
>
> <artifactId>hbase</artifactId>
>
> <version>0.92.1-cdh4.1.2</version>
>
> <exclusions>
>
> <exclusion>
>
> <artifactId>log4j</artifactId>
>
> <groupId>log4j</groupId>
>
> </exclusion>
>
> <exclusion>
>
> <artifactId>slf4j-log4j12</artifactId>
>
> <groupId>org.slf4j</groupId>
>
> </exclusion>
>
> </exclusions>
>
> </dependency>
>
> Regards,
> Shahab
>
>
> On Tue, Aug 27, 2013 at 11:47 AM, jamal sasha <ja...@gmail.com>wrote:
>
>> Hi,
>> For one of my map reduce code I want to use a different version of
>> slf4j jar (1.6.4)
>> But I guess hadoop has a different version of jar in hadoop
>> classpath lib/slf4j-log4j12-1.4.3.jar
>> And when I am trying to run my code, I am gettign this error:
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
>> at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
>> (SLF4JLocationAwareLog.java:159)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:416)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
>> at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>> Similar issue here:
>> http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
>>
>> What should I do?
>> Thanks
>>
>
>
Re: Jar issue
Posted by jamal sasha <ja...@gmail.com>.
I am right now using libjars option.
How do i do what you suggested using that route?
On Tue, Aug 27, 2013 at 8:51 AM, Shahab Yunus <sh...@gmail.com>wrote:
> One idea is, you can use the exclusion property of maven (provided you are
> using that to build your application) while including hadoop dependencies
> and exclude sl4j that is coming within hadoop and then include your own
> sl4j as a separate dependency. Something like this:
>
> <dependency>
>
> <groupId>org.apache.hbase</groupId>
>
> <artifactId>hbase</artifactId>
>
> <version>0.92.1-cdh4.1.2</version>
>
> <exclusions>
>
> <exclusion>
>
> <artifactId>log4j</artifactId>
>
> <groupId>log4j</groupId>
>
> </exclusion>
>
> <exclusion>
>
> <artifactId>slf4j-log4j12</artifactId>
>
> <groupId>org.slf4j</groupId>
>
> </exclusion>
>
> </exclusions>
>
> </dependency>
>
> Regards,
> Shahab
>
>
> On Tue, Aug 27, 2013 at 11:47 AM, jamal sasha <ja...@gmail.com>wrote:
>
>> Hi,
>> For one of my map reduce code I want to use a different version of
>> slf4j jar (1.6.4)
>> But I guess hadoop has a different version of jar in hadoop
>> classpath lib/slf4j-log4j12-1.4.3.jar
>> And when I am trying to run my code, I am gettign this error:
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
>> at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
>> (SLF4JLocationAwareLog.java:159)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:416)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
>> at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>> Similar issue here:
>> http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
>>
>> What should I do?
>> Thanks
>>
>
>
Re: Jar issue
Posted by jamal sasha <ja...@gmail.com>.
I am right now using libjars option.
How do i do what you suggested using that route?
On Tue, Aug 27, 2013 at 8:51 AM, Shahab Yunus <sh...@gmail.com>wrote:
> One idea is, you can use the exclusion property of maven (provided you are
> using that to build your application) while including hadoop dependencies
> and exclude sl4j that is coming within hadoop and then include your own
> sl4j as a separate dependency. Something like this:
>
> <dependency>
>
> <groupId>org.apache.hbase</groupId>
>
> <artifactId>hbase</artifactId>
>
> <version>0.92.1-cdh4.1.2</version>
>
> <exclusions>
>
> <exclusion>
>
> <artifactId>log4j</artifactId>
>
> <groupId>log4j</groupId>
>
> </exclusion>
>
> <exclusion>
>
> <artifactId>slf4j-log4j12</artifactId>
>
> <groupId>org.slf4j</groupId>
>
> </exclusion>
>
> </exclusions>
>
> </dependency>
>
> Regards,
> Shahab
>
>
> On Tue, Aug 27, 2013 at 11:47 AM, jamal sasha <ja...@gmail.com>wrote:
>
>> Hi,
>> For one of my map reduce code I want to use a different version of
>> slf4j jar (1.6.4)
>> But I guess hadoop has a different version of jar in hadoop
>> classpath lib/slf4j-log4j12-1.4.3.jar
>> And when I am trying to run my code, I am gettign this error:
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
>> at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
>> (SLF4JLocationAwareLog.java:159)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:416)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
>> at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>> Similar issue here:
>> http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
>>
>> What should I do?
>> Thanks
>>
>
>
Re: Jar issue
Posted by jamal sasha <ja...@gmail.com>.
I am right now using libjars option.
How do i do what you suggested using that route?
On Tue, Aug 27, 2013 at 8:51 AM, Shahab Yunus <sh...@gmail.com>wrote:
> One idea is, you can use the exclusion property of maven (provided you are
> using that to build your application) while including hadoop dependencies
> and exclude sl4j that is coming within hadoop and then include your own
> sl4j as a separate dependency. Something like this:
>
> <dependency>
>
> <groupId>org.apache.hbase</groupId>
>
> <artifactId>hbase</artifactId>
>
> <version>0.92.1-cdh4.1.2</version>
>
> <exclusions>
>
> <exclusion>
>
> <artifactId>log4j</artifactId>
>
> <groupId>log4j</groupId>
>
> </exclusion>
>
> <exclusion>
>
> <artifactId>slf4j-log4j12</artifactId>
>
> <groupId>org.slf4j</groupId>
>
> </exclusion>
>
> </exclusions>
>
> </dependency>
>
> Regards,
> Shahab
>
>
> On Tue, Aug 27, 2013 at 11:47 AM, jamal sasha <ja...@gmail.com>wrote:
>
>> Hi,
>> For one of my map reduce code I want to use a different version of
>> slf4j jar (1.6.4)
>> But I guess hadoop has a different version of jar in hadoop
>> classpath lib/slf4j-log4j12-1.4.3.jar
>> And when I am trying to run my code, I am gettign this error:
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
>> at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
>> (SLF4JLocationAwareLog.java:159)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:416)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
>> at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>> Similar issue here:
>> http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
>>
>> What should I do?
>> Thanks
>>
>
>
Re: Jar issue
Posted by Shahab Yunus <sh...@gmail.com>.
One idea is, you can use the exclusion property of maven (provided you are
using that to build your application) while including hadoop dependencies
and exclude sl4j that is coming within hadoop and then include your own
sl4j as a separate dependency. Something like this:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.92.1-cdh4.1.2</version>
<exclusions>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
Regards,
Shahab
On Tue, Aug 27, 2013 at 11:47 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> For one of my map reduce code I want to use a different version of slf4j
> jar (1.6.4)
> But I guess hadoop has a different version of jar in hadoop
> classpath lib/slf4j-log4j12-1.4.3.jar
> And when I am trying to run my code, I am gettign this error:
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
> at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
> (SLF4JLocationAwareLog.java:159)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
> at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
> Similar issue here:
> http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
>
> What should I do?
> Thanks
>
Re: Jar issue
Posted by Shahab Yunus <sh...@gmail.com>.
One idea is, you can use the exclusion property of maven (provided you are
using that to build your application) while including hadoop dependencies
and exclude sl4j that is coming within hadoop and then include your own
sl4j as a separate dependency. Something like this:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.92.1-cdh4.1.2</version>
<exclusions>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
Regards,
Shahab
On Tue, Aug 27, 2013 at 11:47 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> For one of my map reduce code I want to use a different version of slf4j
> jar (1.6.4)
> But I guess hadoop has a different version of jar in hadoop
> classpath lib/slf4j-log4j12-1.4.3.jar
> And when I am trying to run my code, I am gettign this error:
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
> at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
> (SLF4JLocationAwareLog.java:159)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
> at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
> Similar issue here:
> http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
>
> What should I do?
> Thanks
>
Re: Jar issue
Posted by Shahab Yunus <sh...@gmail.com>.
One idea is, you can use the exclusion property of maven (provided you are
using that to build your application) while including hadoop dependencies
and exclude sl4j that is coming within hadoop and then include your own
sl4j as a separate dependency. Something like this:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.92.1-cdh4.1.2</version>
<exclusions>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
Regards,
Shahab
On Tue, Aug 27, 2013 at 11:47 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> For one of my map reduce code I want to use a different version of slf4j
> jar (1.6.4)
> But I guess hadoop has a different version of jar in hadoop
> classpath lib/slf4j-log4j12-1.4.3.jar
> And when I am trying to run my code, I am gettign this error:
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
> at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
> (SLF4JLocationAwareLog.java:159)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
> at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
> Similar issue here:
> http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
>
> What should I do?
> Thanks
>
Re: Jar issue
Posted by Shahab Yunus <sh...@gmail.com>.
One idea is, you can use the exclusion property of maven (provided you are
using that to build your application) while including hadoop dependencies
and exclude sl4j that is coming within hadoop and then include your own
sl4j as a separate dependency. Something like this:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>0.92.1-cdh4.1.2</version>
<exclusions>
<exclusion>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
</exclusion>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
Regards,
Shahab
On Tue, Aug 27, 2013 at 11:47 AM, jamal sasha <ja...@gmail.com> wrote:
> Hi,
> For one of my map reduce code I want to use a different version of slf4j
> jar (1.6.4)
> But I guess hadoop has a different version of jar in hadoop
> classpath lib/slf4j-log4j12-1.4.3.jar
> And when I am trying to run my code, I am gettign this error:
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
> at org.apache.commons.logging.impl.SLF4JLocationAwareLog.info
> (SLF4JLocationAwareLog.java:159)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:931)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
> at org.rdf.RdfFormatter.run(RdfFormatter.java:109)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.rdf.RdfFormatter.main(RdfFormatter.java:144)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
> Similar issue here:
> http://stackoverflow.com/questions/10354785/nosuchmethoderror-with-slfj4
>
> What should I do?
> Thanks
>