You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Weiwei Yang (JIRA)" <ji...@apache.org> on 2015/11/19 10:58:11 UTC
[jira] [Commented] (YARN-3556) Yarn REST to submit application
fails with NPE if the element has empty entries
[ https://issues.apache.org/jira/browse/YARN-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013263#comment-15013263 ]
Weiwei Yang commented on YARN-3556:
-----------------------------------
I just spent some time looking at this problem. This is not just for an empty entry in environment, any tag in xml if it is empty, you will get NPE. Yarn leverages protobuf to serialize data, this logic is everywhere, I don't see a good option to fix though I admit the error message is not quite user friendly.
> Yarn REST to submit application fails with NPE if the <environment> element has empty entries
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-3556
> URL: https://issues.apache.org/jira/browse/YARN-3556
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Affects Versions: 2.6.0
> Reporter: Rajesh Kartha
> Assignee: Weiwei Yang
> Priority: Minor
> Attachments: wc-no-cp-NPE.xml
>
>
> I was trying out the YARN REST api to submit applications and ran into a NPE where the <environment> element has empty <entry> tags
> example:
> <am-container-spec>
> <environment>
> <entry>
> </entry>
> </environment>
> <commands>
> <command>hadoop jar /usr/hadoop/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar wordcount /tmp/zkSmoke.out /tmp/REST-WC-4</command>
> </commands>
> </am-container-spec>
> The Exception:
> 2015-04-28 09:34:41,725 WARN resourcemanager.RMAuditLogger (RMAuditLogger.java:logFailure(263)) - USER=dr.who OPERATION=Application Finished - Failed
> TARGET=RMAppManagerRESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1430176860750_0024 failed
> 2 times due to Error launching appattempt_1430176860750_0024_000002.
> Got exception: java.lang.NullPointerException
> at org.apache.hadoop.yarn.proto.YarnProtos$StringStringMapProto$Builder.setKey(YarnProtos.java:44696)
> at org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl$3$1.next(ContainerLaunchContextPBImpl.java:376)
> at org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl$3$1.next(ContainerLaunchContextPBImpl.java:364)
> at com.google.protobuf.AbstractMessageLite$Builder.checkForNullValues(AbstractMessageLite.java:336)
> at com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:323)
> at org.apache.hadoop.yarn.proto.YarnProtos$ContainerLaunchContextProto$Builder.addAllEnvironment(YarnProtos.java:39623)
> at org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl.addEnvToProto(ContainerLaunchContextPBImpl.java:387)
> at org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl.mergeLocalToBuilder(ContainerLaunchContextPBImpl.java:115)
> at org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl.mergeLocalToProto(ContainerLaunchContextPBImpl.java:128)
> at org.apache.hadoop.yarn.api.records.impl.pb.ContainerLaunchContextPBImpl.getProto(ContainerLaunchContextPBImpl.java:70)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl.convertToProtoFormat(StartContainerRequestPBImpl.java:156)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl.mergeLocalToBuilder(StartContainerRequestPBImpl.java:85)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl.mergeLocalToProto(StartContainerRequestPBImpl.java:95)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl.getProto(StartContainerRequestPBImpl.java:57)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.convertToProtoFormat(StartContainersRequestPBImpl.java:137)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.addLocalRequestsToProto(StartContainersRequestPBImpl.java:97)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.mergeLocalToBuilder(StartContainersRequestPBImpl.java:79)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.mergeLocalToProto(StartContainersRequestPBImpl.java:72)
> at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainersRequestPBImpl.getProto(StartContainersRequestPBImpl.java:48)
> at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:93)
> at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:119)
> at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:254)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> . Failing the application. APPID=application_1430176860750_0024
> Would be good to continue the job instead of the NPE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)