You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "John Gordon (JIRA)" <ji...@apache.org> on 2012/06/27 04:42:43 UTC
[jira] [Created] (MAPREDUCE-4377) TaskRunner javaopts parsing
doesn't handle embedded spaces
John Gordon created MAPREDUCE-4377:
--------------------------------------
Summary: TaskRunner javaopts parsing doesn't handle embedded spaces
Key: MAPREDUCE-4377
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4377
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: task-controller
Affects Versions: trunk
Environment: java options containing escaped or non-escaped embedded spaces.
Reporter: John Gordon
TaskRunner::GetVMArgs reads getChildJavaOpts as one space-delimited string, then split is on ' ' and tries to reason on individual options from there. The problem with this approach is that java options may contain embedded spaces in many legitimate cases -- this means it is reasoning on incomplete option strings and cannot do appropriate preprocessing to do things like handle escape characters or matched quotation marks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4377) TaskRunner javaopts parsing
doesn't handle embedded spaces
Posted by "John Gordon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401910#comment-13401910 ]
John Gordon commented on MAPREDUCE-4377:
----------------------------------------
This also means that the code to update the library path doesn't work if there is already a space in the user-specified library path:
for(int i=0; i<javaOptsSplit.length ;i++) {
if(javaOptsSplit[i].startsWith("-Djava.library.path=")) {
javaOptsSplit[i] += SYSTEM_PATH_SEPARATOR + libraryPath;
hasUserLDPath = true;
}
vargs.add(javaOptsSplit[i]);
}
Instead of appending to the end of the string, we are actually appending to the middle of the original user-specified library path (or wherever they had an embedded space).
> TaskRunner javaopts parsing doesn't handle embedded spaces
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4377
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4377
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task-controller
> Affects Versions: trunk
> Environment: java options containing escaped or non-escaped embedded spaces.
> Reporter: John Gordon
>
> TaskRunner::GetVMArgs reads getChildJavaOpts as one space-delimited string, then split is on ' ' and tries to reason on individual options from there. The problem with this approach is that java options may contain embedded spaces in many legitimate cases -- this means it is reasoning on incomplete option strings and cannot do appropriate preprocessing to do things like handle escape characters or matched quotation marks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4377) TaskRunner javaopts parsing
doesn't handle embedded spaces
Posted by "John Gordon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402452#comment-13402452 ]
John Gordon commented on MAPREDUCE-4377:
----------------------------------------
Thanks Robert! I agree it won't be an easy fix and may need some rearchitecture and significant test additions.
> TaskRunner javaopts parsing doesn't handle embedded spaces
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4377
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4377
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task-controller
> Affects Versions: trunk
> Environment: java options containing escaped or non-escaped embedded spaces.
> Reporter: John Gordon
>
> TaskRunner::GetVMArgs reads getChildJavaOpts as one space-delimited string, then split is on ' ' and tries to reason on individual options from there. The problem with this approach is that java options may contain embedded spaces in many legitimate cases -- this means it is reasoning on incomplete option strings and cannot do appropriate preprocessing to do things like handle escape characters or matched quotation marks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4377) TaskRunner javaopts parsing
doesn't handle embedded spaces
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402230#comment-13402230 ]
Robert Joseph Evans commented on MAPREDUCE-4377:
------------------------------------------------
John,
that is very true, and if you can fix it I would be very happy to commit it for you. However, I don't think this is the only place in the code that has problems with embedded spaces. I'm not saying that we should not fix it, we should, just be aware that there be monsters here. Also be aware that there may be some Windows vs. POSIX(bash) issues that you may run into with trying to parse the arguments. Hopefully not too much though.
> TaskRunner javaopts parsing doesn't handle embedded spaces
> ----------------------------------------------------------
>
> Key: MAPREDUCE-4377
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4377
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task-controller
> Affects Versions: trunk
> Environment: java options containing escaped or non-escaped embedded spaces.
> Reporter: John Gordon
>
> TaskRunner::GetVMArgs reads getChildJavaOpts as one space-delimited string, then split is on ' ' and tries to reason on individual options from there. The problem with this approach is that java options may contain embedded spaces in many legitimate cases -- this means it is reasoning on incomplete option strings and cannot do appropriate preprocessing to do things like handle escape characters or matched quotation marks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira