[jira] [Commented] (HADOOP-10123) Invalid NaN values in Hadoop REST API JSON response

["Chen He (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839636#comment-13839636 ] 

Chen He commented on HADOOP-10123:
----------------------------------

Please split this bug into 3 bugs: 1 for Yarn, 1 for MAPRED, 1 for PIG. 
For the Yarn bug, I have following suggestion:

This NaN is caused by customer's InputFormat. 

I propose a solution to fix this problem. 
1) For any float number that is smaller than "0" or "NaN" in progress, we will truncated to "0";
2) For any float number that is larger than "1" in progress, we truncated to "1";
3) For any float number falls into 0 and 1, we keep it;

I set up a filter in ApplicationMasterServices to prevent "illegal" numbers from coming into ResourceManager.

This solution will be ready in New Patch which is named as "Yarn-691-new". Old patch only filters "NaN" in MapReduce jobs.

> Invalid NaN values in Hadoop REST API JSON response
> ---------------------------------------------------
>
>                 Key: HADOOP-10123
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10123
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 0.23.6, 2.0.4-alpha
>            Reporter: Kendall Thrapp
>            Assignee: Chen He
>         Attachments: Yarn-691.patch
>
>
> I've been occasionally coming across instances where Hadoop's Cluster Applications REST API (http://hadoop.apache.org/docs/r0.23.6/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Applications_API) has returned JSON that PHP's json_decode function failed to parse.  I've tracked the syntax error down to the presence of the unquoted word NaN appearing as a value in the JSON.  For example:
> "progress":NaN,
> NaN is not part of the JSON spec, so its presence renders the whole JSON string invalid.  Hadoop needs to return something other than NaN in this case -- perhaps an empty string or the quoted string "NaN".



--
This message was sent by Atlassian JIRA
(v6.1#6144)