You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by Dimple Patel <di...@gmail.com> on 2016/11/22 18:52:08 UTC
App fails due to NoSuchMethodError:
org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
When I launch my app in Apex, in my container logs, I get error log empty but
the app state results as failed and the dt.log is as follows.
tatorrent.stram.StreamingAppMasterService: Application master, appId=1,
clustertimestamp=1479838616829, attemptId=2
2016-11-22 10:23:32,036 ERROR com.datatorrent.stram.StreamingAppMaster:
Exiting Application Master
java.lang.NoSuchMethodError:
org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
at
org.apache.hadoop.mapreduce.util.ConfigUtil.addDeprecatedKeys(ConfigUtil.java:54)
at
org.apache.hadoop.mapreduce.util.ConfigUtil.loadResources(ConfigUtil.java:42)
at org.apache.hadoop.mapred.JobConf.<clinit>(JobConf.java:118)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at
org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1659)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:91)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:235)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:214)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:669)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:571)
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:442)
at
com.datatorrent.common.util.FSStorageAgent.<init>(FSStorageAgent.java:84)
at
com.datatorrent.common.util.AsyncFSStorageAgent.<init>(AsyncFSStorageAgent.java:64)
at
com.datatorrent.common.util.AsyncFSStorageAgent.readResolve(AsyncFSStorageAgent.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1807)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at java.util.HashMap.readObject(HashMap.java:1180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1897)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1997)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1921)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1997)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1921)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at
com.datatorrent.stram.plan.logical.LogicalPlan.read(LogicalPlan.java:2331)
at
com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:542)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:102)
Initially I was getting Multiple slf4j bindings were found on the class
path, which I solved using this
<http://www.slf4j.org/codes.html#multiple_bindings> link but now my app
fails with this log. Many solutions available on stackoverflow are not
useful as I am not using hive or hortonworks. I am using only Apache Apex
and datatorrent RTS to create my app. I don't know where else to look to
resolve this error. Some direction to resolve this will be appreciated.
Thanks,
Dimple
--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/App-fails-due-to-NoSuchMethodError-org-apache-hadoop-conf-Configuration-addDeprecations-Lorg-apache-V-tp1160.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.
Re: App fails due to NoSuchMethodError: org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
Posted by Munagala Ramanath <ra...@datatorrent.com>.
Usually this sort of error is a symptom of including hadoop jars in your
application package.
Please make sure this is not happening. This link has some info:
http://docs.datatorrent.com/troubleshooting/#hadoop-dependencies-conflicts
If you run "jar tvf target/*.apa | grep hadoop" you'll see which jars are
being included
Ram
On Tue, Nov 22, 2016 at 10:52 AM, Dimple Patel <di...@gmail.com>
wrote:
> When I launch my app in Apex, in my container logs, I get error log empty
> but
> the app state results as failed and the dt.log is as follows.
>
> tatorrent.stram.StreamingAppMasterService: Application master, appId=1,
> clustertimestamp=1479838616829, attemptId=2
> 2016-11-22 10:23:32,036 ERROR com.datatorrent.stram.StreamingAppMaster:
> Exiting Application Master
> java.lang.NoSuchMethodError:
> org.apache.hadoop.conf.Configuration.addDeprecations(
> [Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
> at
> org.apache.hadoop.mapreduce.util.ConfigUtil.addDeprecatedKeys(ConfigUtil.
> java:54)
> at
> org.apache.hadoop.mapreduce.util.ConfigUtil.loadResources(
> ConfigUtil.java:42)
> at org.apache.hadoop.mapred.JobConf.<clinit>(JobConf.java:118)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:274)
> at
> org.apache.hadoop.conf.Configuration.getClassByNameOrNull(
> Configuration.java:1659)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:91)
> at org.apache.hadoop.util.ReflectionUtils.setConf(
> ReflectionUtils.java:75)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(
> ReflectionUtils.java:133)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
> at
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(
> Groups.java:182)
> at
> org.apache.hadoop.security.UserGroupInformation.initialize(
> UserGroupInformation.java:235)
> at
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(
> UserGroupInformation.java:214)
> at
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(
> UserGroupInformation.java:669)
> at
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(
> UserGroupInformation.java:571)
> at org.apache.hadoop.fs.FileContext.getFileContext(
> FileContext.java:442)
> at
> com.datatorrent.common.util.FSStorageAgent.<init>(FSStorageAgent.java:84)
> at
> com.datatorrent.common.util.AsyncFSStorageAgent.<init>(
> AsyncFSStorageAgent.java:64)
> at
> com.datatorrent.common.util.AsyncFSStorageAgent.readResolve(
> AsyncFSStorageAgent.java:151)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at java.io.ObjectStreamClass.invokeReadResolve(
> ObjectStreamClass.java:1104)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1807)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.
> java:1350)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.
> java:370)
> at java.util.HashMap.readObject(HashMap.java:1180)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at java.io.ObjectStreamClass.invokeReadObject(
> ObjectStreamClass.java:1017)
> at java.io.ObjectInputStream.readSerialData(
> ObjectInputStream.java:1897)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.
> java:1350)
> at java.io.ObjectInputStream.defaultReadFields(
> ObjectInputStream.java:1997)
> at java.io.ObjectInputStream.readSerialData(
> ObjectInputStream.java:1921)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.
> java:1350)
> at java.io.ObjectInputStream.defaultReadFields(
> ObjectInputStream.java:1997)
> at java.io.ObjectInputStream.readSerialData(
> ObjectInputStream.java:1921)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.
> java:1350)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.
> java:370)
> at
> com.datatorrent.stram.plan.logical.LogicalPlan.read(LogicalPlan.java:2331)
> at
> com.datatorrent.stram.StreamingAppMasterService.serviceInit(
> StreamingAppMasterService.java:542)
> at org.apache.hadoop.service.AbstractService.init(
> AbstractService.java:163)
> at
> com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:102)
>
> Initially I was getting Multiple slf4j bindings were found on the class
> path, which I solved using this
> <http://www.slf4j.org/codes.html#multiple_bindings> link but now my app
> fails with this log. Many solutions available on stackoverflow are not
> useful as I am not using hive or hortonworks. I am using only Apache Apex
> and datatorrent RTS to create my app. I don't know where else to look to
> resolve this error. Some direction to resolve this will be appreciated.
>
> Thanks,
> Dimple
>
>
>
> --
> View this message in context: http://apache-apex-users-list.
> 78494.x6.nabble.com/App-fails-due-to-NoSuchMethodError-org-
> apache-hadoop-conf-Configuration-addDeprecations-Lorg-apache-V-tp1160.html
> Sent from the Apache Apex Users list mailing list archive at Nabble.com.
>
Re: App fails due to NoSuchMethodError:
org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
Posted by Dimple Patel <di...@gmail.com>.
Apex version - 3.5.0-dt20160916
Hadoop version - 2.7.1
--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/App-fails-due-to-NoSuchMethodError-org-apache-hadoop-conf-Configuration-addDeprecations-Lorg-apache-V-tp1160p1163.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.
Re: App fails due to NoSuchMethodError:
org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
Posted by Sanjay Pujare <sa...@datatorrent.com>.
What version(s) of Apex are you using? Recently Hadoop dependencies were upgraded to use Hadoop 2.6.* but I am not sure if you are running into that issue…
On 11/22/16, 10:52 AM, "Dimple Patel" <di...@gmail.com> wrote:
When I launch my app in Apex, in my container logs, I get error log empty but
the app state results as failed and the dt.log is as follows.
tatorrent.stram.StreamingAppMasterService: Application master, appId=1,
clustertimestamp=1479838616829, attemptId=2
2016-11-22 10:23:32,036 ERROR com.datatorrent.stram.StreamingAppMaster:
Exiting Application Master
java.lang.NoSuchMethodError:
org.apache.hadoop.conf.Configuration.addDeprecations([Lorg/apache/hadoop/conf/Configuration$DeprecationDelta;)V
at
org.apache.hadoop.mapreduce.util.ConfigUtil.addDeprecatedKeys(ConfigUtil.java:54)
at
org.apache.hadoop.mapreduce.util.ConfigUtil.loadResources(ConfigUtil.java:42)
at org.apache.hadoop.mapred.JobConf.<clinit>(JobConf.java:118)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at
org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1659)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:91)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:235)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:214)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:669)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:571)
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:442)
at
com.datatorrent.common.util.FSStorageAgent.<init>(FSStorageAgent.java:84)
at
com.datatorrent.common.util.AsyncFSStorageAgent.<init>(AsyncFSStorageAgent.java:64)
at
com.datatorrent.common.util.AsyncFSStorageAgent.readResolve(AsyncFSStorageAgent.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1807)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at java.util.HashMap.readObject(HashMap.java:1180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1897)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1997)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1921)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1997)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1921)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at
com.datatorrent.stram.plan.logical.LogicalPlan.read(LogicalPlan.java:2331)
at
com.datatorrent.stram.StreamingAppMasterService.serviceInit(StreamingAppMasterService.java:542)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:102)
Initially I was getting Multiple slf4j bindings were found on the class
path, which I solved using this
<http://www.slf4j.org/codes.html#multiple_bindings> link but now my app
fails with this log. Many solutions available on stackoverflow are not
useful as I am not using hive or hortonworks. I am using only Apache Apex
and datatorrent RTS to create my app. I don't know where else to look to
resolve this error. Some direction to resolve this will be appreciated.
Thanks,
Dimple
--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/App-fails-due-to-NoSuchMethodError-org-apache-hadoop-conf-Configuration-addDeprecations-Lorg-apache-V-tp1160.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.