You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Rob Blah <tm...@gmail.com> on 2014/08/21 13:01:17 UTC

Yarn 2.4.1 NoSuchMethodError for DatanodeProtocolProtos, while copying data to HDFS.

Hi

We are using Yarn 2.4.1 build from svn as follows "package -Pdist -Dtar
-DskipTests". Build is successful and YARN is starting also successful.
However when we are trying to copy data to HDFS (copyFromLocal) we have the
following exception:

Namenode.log
java.lang.NoSuchMethodError:
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
        at
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)

We are using protocolbuf 2.5.0. Any ideas?

regards
tmp

Re: Yarn 2.4.1 NoSuchMethodError for DatanodeProtocolProtos, while copying data to HDFS.

Posted by Rob Blah <tm...@gmail.com>.
Sorry I forgot to add Datanode logs:
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: RemoteException in
offerService
org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
        at
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)

        at org.apache.hadoop.ipc.Client.call(Client.java:1411)
        at org.apache.hadoop.ipc.Client.call(Client.java:1364)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
        at com.sun.proxy.$Proxy14.blockReport(Unknown Source)
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.blockReport(DatanodeProtocolClientSideTranslatorPB.java:175)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.blockReport(BPServiceActor.java:488)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:711)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846)
        at java.lang.Thread.run(Thread.java:701)



2014-08-21 13:01 GMT+02:00 Rob Blah <tm...@gmail.com>:

> Hi
>
> We are using Yarn 2.4.1 build from svn as follows "package -Pdist -Dtar
> -DskipTests". Build is successful and YARN is starting also successful.
> However when we are trying to copy data to HDFS (copyFromLocal) we have the
> following exception:
>
> Namenode.log
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
>         at
> org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
>         at
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
>         at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:416)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)
>
> We are using protocolbuf 2.5.0. Any ideas?
>
> regards
> tmp
>

Re: Yarn 2.4.1 NoSuchMethodError for DatanodeProtocolProtos, while copying data to HDFS.

Posted by Rob Blah <tm...@gmail.com>.
Sorry I forgot to add Datanode logs:
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: RemoteException in
offerService
org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
        at
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)

        at org.apache.hadoop.ipc.Client.call(Client.java:1411)
        at org.apache.hadoop.ipc.Client.call(Client.java:1364)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
        at com.sun.proxy.$Proxy14.blockReport(Unknown Source)
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.blockReport(DatanodeProtocolClientSideTranslatorPB.java:175)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.blockReport(BPServiceActor.java:488)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:711)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846)
        at java.lang.Thread.run(Thread.java:701)



2014-08-21 13:01 GMT+02:00 Rob Blah <tm...@gmail.com>:

> Hi
>
> We are using Yarn 2.4.1 build from svn as follows "package -Pdist -Dtar
> -DskipTests". Build is successful and YARN is starting also successful.
> However when we are trying to copy data to HDFS (copyFromLocal) we have the
> following exception:
>
> Namenode.log
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
>         at
> org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
>         at
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
>         at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:416)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)
>
> We are using protocolbuf 2.5.0. Any ideas?
>
> regards
> tmp
>

Re: Yarn 2.4.1 NoSuchMethodError for DatanodeProtocolProtos, while copying data to HDFS.

Posted by Rob Blah <tm...@gmail.com>.
Sorry I forgot to add Datanode logs:
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: RemoteException in
offerService
org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
        at
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)

        at org.apache.hadoop.ipc.Client.call(Client.java:1411)
        at org.apache.hadoop.ipc.Client.call(Client.java:1364)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
        at com.sun.proxy.$Proxy14.blockReport(Unknown Source)
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.blockReport(DatanodeProtocolClientSideTranslatorPB.java:175)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.blockReport(BPServiceActor.java:488)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:711)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846)
        at java.lang.Thread.run(Thread.java:701)



2014-08-21 13:01 GMT+02:00 Rob Blah <tm...@gmail.com>:

> Hi
>
> We are using Yarn 2.4.1 build from svn as follows "package -Pdist -Dtar
> -DskipTests". Build is successful and YARN is starting also successful.
> However when we are trying to copy data to HDFS (copyFromLocal) we have the
> following exception:
>
> Namenode.log
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
>         at
> org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
>         at
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
>         at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:416)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)
>
> We are using protocolbuf 2.5.0. Any ideas?
>
> regards
> tmp
>

Re: Yarn 2.4.1 NoSuchMethodError for DatanodeProtocolProtos, while copying data to HDFS.

Posted by Rob Blah <tm...@gmail.com>.
Sorry I forgot to add Datanode logs:
WARN org.apache.hadoop.hdfs.server.datanode.DataNode: RemoteException in
offerService
org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
        at
org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:416)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)

        at org.apache.hadoop.ipc.Client.call(Client.java:1411)
        at org.apache.hadoop.ipc.Client.call(Client.java:1364)
        at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
        at com.sun.proxy.$Proxy14.blockReport(Unknown Source)
        at
org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolClientSideTranslatorPB.blockReport(DatanodeProtocolClientSideTranslatorPB.java:175)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.blockReport(BPServiceActor.java:488)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:711)
        at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846)
        at java.lang.Thread.run(Thread.java:701)



2014-08-21 13:01 GMT+02:00 Rob Blah <tm...@gmail.com>:

> Hi
>
> We are using Yarn 2.4.1 build from svn as follows "package -Pdist -Dtar
> -DskipTests". Build is successful and YARN is starting also successful.
> However when we are trying to copy data to HDFS (copyFromLocal) we have the
> following exception:
>
> Namenode.log
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$StorageBlockReportProto.getStorage()Lorg/apache/hadoop/hdfs/protocol/proto/DatanodeProtocolProtos$DatanodeStorageProto;
>         at
> org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:148)
>         at
> org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26300)
>         at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:607)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:416)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)
>
> We are using protocolbuf 2.5.0. Any ideas?
>
> regards
> tmp
>