You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2011/07/19 23:26:57 UTC
[jira] [Created] (HADOOP-7476) task-controller can drop last char
from config file
task-controller can drop last char from config file
---------------------------------------------------
Key: HADOOP-7476
URL: https://issues.apache.org/jira/browse/HADOOP-7476
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.20.203.0
Reporter: Allen Wittenauer
Priority: Trivial
It looks as though task-controller's configuration file reader assumes that the output of getline() always ends with \n\0. This assumption does not appear to be safe. See comments for more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-7476) task-controller can drop last char
from config file
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers resolved HADOOP-7476.
------------------------------------
Resolution: Duplicate
> task-controller can drop last char from config file
> ---------------------------------------------------
>
> Key: HADOOP-7476
> URL: https://issues.apache.org/jira/browse/HADOOP-7476
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.20.203.0
> Reporter: Allen Wittenauer
> Priority: Trivial
>
> It looks as though task-controller's configuration file reader assumes that the output of getline() always ends with \n\0. This assumption does not appear to be safe. See comments for more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7476) task-controller can drop last char
from config file
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067979#comment-13067979 ]
Aaron T. Myers commented on HADOOP-7476:
----------------------------------------
Is this not a dupe of MAPREDUCE-2377 ?
> task-controller can drop last char from config file
> ---------------------------------------------------
>
> Key: HADOOP-7476
> URL: https://issues.apache.org/jira/browse/HADOOP-7476
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.20.203.0
> Reporter: Allen Wittenauer
> Priority: Trivial
>
> It looks as though task-controller's configuration file reader assumes that the output of getline() always ends with \n\0. This assumption does not appear to be safe. See comments for more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7476) task-controller can drop last char
from config file
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067988#comment-13067988 ]
Allen Wittenauer commented on HADOOP-7476:
------------------------------------------
sure is, missed that one. thanks.
> task-controller can drop last char from config file
> ---------------------------------------------------
>
> Key: HADOOP-7476
> URL: https://issues.apache.org/jira/browse/HADOOP-7476
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.20.203.0
> Reporter: Allen Wittenauer
> Priority: Trivial
>
> It looks as though task-controller's configuration file reader assumes that the output of getline() always ends with \n\0. This assumption does not appear to be safe. See comments for more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7476) task-controller can drop last char
from config file
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068073#comment-13068073 ]
Allen Wittenauer commented on HADOOP-7476:
------------------------------------------
OS X and Solaris. As usual, this typically means removing the GNU-only crud*. At this point, my task-controller uses fgetln() instead of getline(). Since I'm lazy, it is easier to find code we can import that implements fgetln() in a portable fashion than getline(). (Altho if getline() is present w/out fgetln(), I've got a wrapper that implements fgetln() with getline()).
* Technically, getline() was added to super-recent POSIX, but none of the platforms that I have access to have that other than glibc-based machines. So it isn't that portable yet. :(
> task-controller can drop last char from config file
> ---------------------------------------------------
>
> Key: HADOOP-7476
> URL: https://issues.apache.org/jira/browse/HADOOP-7476
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.20.203.0
> Reporter: Allen Wittenauer
> Priority: Trivial
>
> It looks as though task-controller's configuration file reader assumes that the output of getline() always ends with \n\0. This assumption does not appear to be safe. See comments for more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7476) task-controller can drop last char
from config file
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067995#comment-13067995 ]
Aaron T. Myers commented on HADOOP-7476:
----------------------------------------
No problem.
Out of curiosity, to what are you porting {{task-controller}}?
> task-controller can drop last char from config file
> ---------------------------------------------------
>
> Key: HADOOP-7476
> URL: https://issues.apache.org/jira/browse/HADOOP-7476
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.20.203.0
> Reporter: Allen Wittenauer
> Priority: Trivial
>
> It looks as though task-controller's configuration file reader assumes that the output of getline() always ends with \n\0. This assumption does not appear to be safe. See comments for more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7476) task-controller can drop last char
from config file
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067978#comment-13067978 ]
Allen Wittenauer commented on HADOOP-7476:
------------------------------------------
While working on porting task-controller, I ran into getline():
{code}
size_read = getline(&line,&linesize,conf_file);
//feof returns true only after we read past EOF.
//so a file with no new line, at last can reach this place
//if size_read returns negative check for eof condition
if (size_read == -1) {
if(!feof(conf_file)){
fprintf(LOGFILE, "getline returned error.\n");
exit(INVALID_CONFIG_FILE);
}else {
free(line);
break;
}
}
//trim the ending new line
line[strlen(line)-1] = '\0';
//comment line
{code}
My read of this code says that we always remove the last character of the buffer prior to the null termination. In the vast majority of cases, this should be \N. However, getline() doesn't appear to guarantee this:
"The buffer is null-terminated and includes the newline character, if one was found."
If the configuration file was built in such a way that it does not end with a newline, it will chop off the last character.
> task-controller can drop last char from config file
> ---------------------------------------------------
>
> Key: HADOOP-7476
> URL: https://issues.apache.org/jira/browse/HADOOP-7476
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
> Affects Versions: 0.20.203.0
> Reporter: Allen Wittenauer
> Priority: Trivial
>
> It looks as though task-controller's configuration file reader assumes that the output of getline() always ends with \n\0. This assumption does not appear to be safe. See comments for more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira