You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2009/04/18 01:20:15 UTC
[jira] Created: (CHUKWA-155) Job History status arrive out of order
causing the status to update incorrectly.
Job History status arrive out of order causing the status to update incorrectly.
--------------------------------------------------------------------------------
Key: CHUKWA-155
URL: https://issues.apache.org/jira/browse/CHUKWA-155
Project: Hadoop Chukwa
Issue Type: Bug
Components: data collection, Data Processors
Environment: Redhat 5.1, Java 6
Reporter: Eric Yang
Assignee: Jerome Boulon
Priority: Critical
Job history contains lines like:
Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
...
Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-155) Job History status arrive out of order
causing the status to update incorrectly.
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-155:
-------------------------
Attachment: chukwa-155.patch
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700389#action_12700389 ]
Cheng commented on CHUKWA-155:
------------------------------
We could ask Hadoop team to add time stamp. But as a generic data collecting system, we should have the ability to deal with any kind of source data.
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Critical
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702802#action_12702802 ]
Hudson commented on CHUKWA-155:
-------------------------------
Integrated in Chukwa-trunk #8 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/8/])
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Fix For: 0.1.2
>
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716995#action_12716995 ]
Hudson commented on CHUKWA-155:
-------------------------------
Integrated in Chukwa-trunk #45 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/45/])
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Fix For: 0.1.2
>
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701339#action_12701339 ]
Eric Yang commented on CHUKWA-155:
----------------------------------
+1 with Jerome's proposal.
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-155) Job History status arrive out of order
causing the status to update incorrectly.
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-155:
-------------------------
Attachment: chukwa-155.patch
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng reassigned CHUKWA-155:
----------------------------
Assignee: Cheng (was: Jerome Boulon)
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701343#action_12701343 ]
Cheng commented on CHUKWA-155:
------------------------------
new patch submitted. if finish_time found, add a new key JOB_FINAL_STATUS.
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700938#action_12700938 ]
Jerome Boulon commented on CHUKWA-155:
--------------------------------------
+1 on asking Hadoop team to add time stamp since we want to do some time based analytic.
Demux is able to deal with any kind of data but if there's some rules.
It's the parser responsibility to provide
- provide a time stamp, if any, use the default one provided by the Collector at the Chunk level
- a key that will group information together according to the data usage
Regarding the case where the data does not contain any time stamp the system will do a best effort to partition the data based on collector time stamp but the parser could/should guarantee the order by specifying a key that contains the SeqId + offset within the same chunk.
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Jerome Boulon
> Priority: Critical
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-155) Job History status arrive out of order
causing the status to update incorrectly.
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-155:
-------------------------
Status: Patch Available (was: Open)
Added timestamp for job log.
If JOB_STATUS == "SUCCESS", ChukwaRecord will have JOB_STATUS_SUCCESS="timestamp"
If JOB_STATUS == "PREP", ChukwaRecord will have JOB_STATUS_PREP="timestamp"
......
If JOB_STATUS == null, ChukwaRecord will have JOB_STATUS_null="timestamp"
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-155) Job History status arrive out of order
causing the status to update incorrectly.
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-155:
-----------------------------
Resolution: Fixed
Fix Version/s: 0.1.2
Status: Resolved (was: Patch Available)
I just committed this, thanks Cheng.
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Fix For: 0.1.2
>
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701712#action_12701712 ]
Eric Yang commented on CHUKWA-155:
----------------------------------
+1 looks good.
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-155) Job History status arrive out of order
causing the status to update incorrectly.
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-155:
-------------------------
Attachment: (was: chukwa-155.patch)
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-155) Job History status arrive out of
order causing the status to update incorrectly.
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701330#action_12701330 ]
Jerome Boulon commented on CHUKWA-155:
--------------------------------------
my 2 cents ...
Or we could have a more generic way:
The problem is that the mysql Job table does not contains columns for more than one state.
What we care about in this table is the final Job state.
So why not create one additional key JOB_FINAL_STATE that will contain the final Job state, aka, success, killed or failed.
the "JOB_FINAL_STATE" 's key could be created at the same time as the finish-time parsing from JobHistory.
For example:
Job JOBID="job_200903310541_1200" FINISH_TIME="1238528943585" JOB_STATUS="SUCCESS" will give JOB_FINAL_STATE="SUCCESS"
Then all others JOB_STATUS should remain unchanged, aka JOB_STATUS="WHAT_EVER_THE_VALUE_IS", timestamp will be what ever is available at that time.
This will give us the transition-states table.
> Job History status arrive out of order causing the status to update incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.