You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Harsh J (JIRA)" <ji...@apache.org> on 2013/04/08 10:47:15 UTC
[jira] [Updated] (YARN-555) ContainerLaunchContext is buggy when it
comes to setter methods on a new instance
[ https://issues.apache.org/jira/browse/YARN-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated YARN-555:
-------------------------
Description:
If you look at the API of ContainerLaunchContext, its got setter methods, such as for setResource, setCommands, etc…:
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/api/records/ContainerLaunchContext.html#setCommands(java.util.List)
However, there's certain things broken in its use here that am trying to understand. Let me explain with some code context:
1. I initialize a proper CLC for an ApplicationSubmissionContext (appContext).
{code}
ContainerLaunchContext appMasterLaunchContext = Records.newRecord(ContainerLaunchContext.class);
appContext.setAMContainerSpec(appMasterLaunchContext);
{code}
2. I create a resource request of 130 MB, as applicationMasterResource, and try to set it into the CLC via:
{code}
appContext.getAMContainerSpec().setResource(applicationMasterResource);
{code}
3. This works OK. If I query it back now, it returns 130 for a {{getMemory()}} call.
4. So I attempt to do the same with setCommands/setEnvironment/etc., all of which fail to mutate cause the check in CLC's implementation class disregards whatever I try to set for some reason.
Edit: It seems like the issue is that when I do a appContext.getAMContainerSpec().getLocalResources() or similar call to get existing initialized data structures to populate further on, what I really get underneath is a silently non-mutative data structure that I can call .put or .add on, but it won't really reflect it.
was:
If you look at the API of ContainerLaunchContext, its got setter methods, such as for setResource, setCommands, etc…:
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/api/records/ContainerLaunchContext.html#setCommands(java.util.List)
However, there's certain things broken in its use here that am trying to understand. Let me explain with some code context:
1. I initialize a proper CLC for an ApplicationSubmissionContext (appContext).
{code}
ContainerLaunchContext appMasterLaunchContext = Records.newRecord(ContainerLaunchContext.class);
appContext.setAMContainerSpec(appMasterLaunchContext);
{code}
2. I create a resource request of 130 MB, as applicationMasterResource, and try to set it into the CLC via:
{code}
appContext.getAMContainerSpec().setResource(applicationMasterResource);
{code}
3. This works OK. If I query it back now, it returns 130 for a {{getMemory()}} call.
4. So I attempt to do the same with setCommands/setEnvironment/etc., all of which fail to mutate cause the check in CLC's implementation class disregards whatever I try to set. This is cause of these null checks which keep passing:
{code}
// ContainerLaunchContextPBImpl.java
@Override
public void setCommands(final List<String> commands) {
if (commands == null)
return;
initCommands();
this.commands.clear();
this.commands.addAll(commands);
}
{code}
This is rather non intuitive as a check. If I am to set something, setting it should take place. If it is null, do not return but instead set whats provided? I'm not even sure why that null check exists - it seems to do so from the start of time.
However, {{setResource(…)}} works pretty fine, as the call has no such odd check:
{code}
@Override
public void setResource(Resource resource) {
maybeInitBuilder();
if (resource == null)
builder.clearResource();
this.resource = resource;
}
{code}
> ContainerLaunchContext is buggy when it comes to setter methods on a new instance
> ---------------------------------------------------------------------------------
>
> Key: YARN-555
> URL: https://issues.apache.org/jira/browse/YARN-555
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
> Affects Versions: 2.0.3-alpha
> Reporter: Harsh J
> Priority: Minor
>
> If you look at the API of ContainerLaunchContext, its got setter methods, such as for setResource, setCommands, etc…:
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/api/records/ContainerLaunchContext.html#setCommands(java.util.List)
> However, there's certain things broken in its use here that am trying to understand. Let me explain with some code context:
> 1. I initialize a proper CLC for an ApplicationSubmissionContext (appContext).
> {code}
> ContainerLaunchContext appMasterLaunchContext = Records.newRecord(ContainerLaunchContext.class);
> appContext.setAMContainerSpec(appMasterLaunchContext);
> {code}
> 2. I create a resource request of 130 MB, as applicationMasterResource, and try to set it into the CLC via:
> {code}
> appContext.getAMContainerSpec().setResource(applicationMasterResource);
> {code}
> 3. This works OK. If I query it back now, it returns 130 for a {{getMemory()}} call.
> 4. So I attempt to do the same with setCommands/setEnvironment/etc., all of which fail to mutate cause the check in CLC's implementation class disregards whatever I try to set for some reason.
> Edit: It seems like the issue is that when I do a appContext.getAMContainerSpec().getLocalResources() or similar call to get existing initialized data structures to populate further on, what I really get underneath is a silently non-mutative data structure that I can call .put or .add on, but it won't really reflect it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira