You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Ari Rabkin (JIRA)" <ji...@apache.org> on 2009/12/22 21:52:29 UTC
[jira] Reopened: (CHUKWA-421) new logfile rotation detection scheme
[ https://issues.apache.org/jira/browse/CHUKWA-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin reopened CHUKWA-421:
-------------------------------
There's a subtle bug in how the new adaptor writes checkpoints. The last-mod date in the checkpoint reflects data SENT, not data WRITTEN. So after a collector crash, this adaptor will resume at the wrong place. Fix is for the adaptor to keep a mapping from send offset to last-mod-date, listen to commit reports, and write accordingly.
If other adaptors start needing to do this, we might need to pull this functionality out into a utility class.
> new logfile rotation detection scheme
> -------------------------------------
>
> Key: CHUKWA-421
> URL: https://issues.apache.org/jira/browse/CHUKWA-421
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Components: data collection
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-421.patch
>
>
> Currently, we keep files open in order to detect rotation. This is wasteful of file handles, doesn't work correctly across reboots, and I think we can do better.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.