[jira] [Commented] (SAMZA-851) Hello-samza doesn't work on CDH

["Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140316#comment-15140316 ] 

Yi Pan (Data Infrastructure) commented on SAMZA-851:
----------------------------------------------------

[~staslos], I have applied the patch to both latest and master branches and successfully build w/ and w/o CDH profile. Unfortunately, I don't have a CDH installation and couldn't verify directly via deploying to CDH. Could you confirm that you have deployed the patched hello-samza to CDH 5.4 cluster?

Thanks!

> Hello-samza doesn't work on CDH
> -------------------------------
>
>                 Key: SAMZA-851
>                 URL: https://issues.apache.org/jira/browse/SAMZA-851
>             Project: Samza
>          Issue Type: Bug
>          Components: hello-samza
>    Affects Versions: 0.10.0, 0.10.1
>            Reporter: Stanislav Los
>            Assignee: Stanislav Los
>         Attachments: 0001-Maven-profile-for-the-CDH-compatible-build-replacing.patch
>
>
> Hello-samza 0.10.0 doesn't work on CDH (tested on CDH5.4.0) due to incompatible Hadoop jars.
> It runs fine on local grid. But when running on cluster I can see it talks fine to YARN's  ResourceManager and Kafka, get's resources, but fails to start container. At first, I thought it's because CDH5.4 uses YARN 2.6.0, while Samza uses 2.6.1, so I recompiled Samza with YARN 2.6.0 and re-assambled hello-samza. No luck so far. Also, I thought it's because of different protobuf version, but I checked, it's the same in CDH and Samza.
> Followed http://samza.apache.org/learn/tutorials/0.10/deploy-samza-to-CDH.html to deploy.
> See full logs: 
> Samza Deploy Script output http://pastebin.com/0K8HVMzt
> YARN logs http://pastebin.com/UvNp6240
> An error is:
> java version "1.7.0_80"
> Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
> com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
> 	at com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:89)
> 	at com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:108)
> 	at org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$AMRMTokenIdentifierProto.<init>(YarnSecurityTokenProtos.java:1078)
> 	at org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$AMRMTokenIdentifierProto.<init>(YarnSecurityTokenProtos.java:1042)
> 	at org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$AMRMTokenIdentifierProto$1.parsePartialFrom(YarnSecurityTokenProtos.java:1138)
> 	at org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$AMRMTokenIdentifierProto$1.parsePartialFrom(YarnSecurityTokenProtos.java:1133)
> 	at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
> 	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
> 	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:223)
> 	at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> 	at org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$AMRMTokenIdentifierProto.parseFrom(YarnSecurityTokenProtos.java:1307)
> 	at org.apache.hadoop.yarn.security.AMRMTokenIdentifier.readFields(AMRMTokenIdentifier.java:81)
> 	at org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:142)
> 	at org.apache.samza.job.yarn.ContainerUtil.startContainer(ContainerUtil.java:181)
> 	at org.apache.samza.job.yarn.ContainerUtil.runContainer(ContainerUtil.java:119)
> 	at org.apache.samza.job.yarn.ContainerAllocator.run(ContainerAllocator.java:65)
> 	at java.lang.Thread.run(Thread.java:745)
> Exception in thread "Container Allocator Thread" org.apache.samza.SamzaException: IO Exception when writing credentials to output buffer
> 	at org.apache.samza.job.yarn.ContainerUtil.startContainer(ContainerUtil.java:190)
> 	at org.apache.samza.job.yarn.ContainerUtil.runContainer(ContainerUtil.java:119)
> 	at org.apache.samza.job.yarn.ContainerAllocator.run(ContainerAllocator.java:65)
> 	at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)