You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@chukwa.apache.org by "Cheng (JIRA)" <ji...@apache.org> on 2009/04/18 02:22:15 UTC

[jira] Commented: (CHUKWA-155) Job History status arrive out of order causing the status to update incorrectly.

    [ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700389#action_12700389 ] 

Cheng commented on CHUKWA-155:
------------------------------

We could ask Hadoop team to add time stamp. But as a generic data collecting system, we should have the ability to deal with any kind of source data.

> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
>                 Key: CHUKWA-155
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-155
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: data collection, Data Processors
>         Environment: Redhat 5.1, Java 6
>            Reporter: Eric Yang
>            Assignee: Jerome Boulon
>            Priority: Critical
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order.  The database is updated with status "RUNNING" instead of "SUCCESS".  
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.