You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2009/06/25 02:36:07 UTC
[jira] Created: (CHUKWA-332) Remove first offset tracker from
CharFileTailingAdaptorUTF8NewLineEscaped adaptor
Remove first offset tracker from CharFileTailingAdaptorUTF8NewLineEscaped adaptor
---------------------------------------------------------------------------------
Key: CHUKWA-332
URL: https://issues.apache.org/jira/browse/CHUKWA-332
Project: Hadoop Chukwa
Issue Type: Improvement
Components: data collection
Affects Versions: 0.3.0
Environment: Redhat EL 5.1, Java 6
Reporter: Eric Yang
Fix For: 0.3.0
CharFileTailingAdaptorUTF8NewLineEscaped is currently taking Starting offset of the file, and current streaming offset of the file. The original idea of using starting offset as a unique sequence tracker to track the unique record for the life time of the stream. However that idea didn't pan out. Hence, we should remove the first offset to avoid confusion. The current streaming offset is sufficient to start tailing file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-332) Remove first offset tracker from
CharFileTailingAdaptorUTF8NewLineEscaped adaptor
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723832#action_12723832 ]
Ari Rabkin commented on CHUKWA-332:
-----------------------------------
Also. I believe this applies to all the FileTalingAdaptors, not just UTF8NewLineEscaped.
> Remove first offset tracker from CharFileTailingAdaptorUTF8NewLineEscaped adaptor
> ---------------------------------------------------------------------------------
>
> Key: CHUKWA-332
> URL: https://issues.apache.org/jira/browse/CHUKWA-332
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Components: data collection
> Affects Versions: 0.3.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Fix For: 0.3.0
>
>
> CharFileTailingAdaptorUTF8NewLineEscaped is currently taking Starting offset of the file, and current streaming offset of the file. The original idea of using starting offset as a unique sequence tracker to track the unique record for the life time of the stream. However that idea didn't pan out. Hence, we should remove the first offset to avoid confusion. The current streaming offset is sufficient to start tailing file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-332) Remove first offset tracker from
CharFileTailingAdaptorUTF8NewLineEscaped adaptor
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723831#action_12723831 ]
Ari Rabkin commented on CHUKWA-332:
-----------------------------------
Hrm. I just want to understand how we currently fix the problem that parameter was there to address.
- Suppose I have a file, foo.txt, containing three chars "abcd".
- An adaptor looks at the file, and sends a new chunk, containing "abcd", ending at offset 4.
- The file then rotates, and now contains "efg". What should the adaptor send?
> Remove first offset tracker from CharFileTailingAdaptorUTF8NewLineEscaped adaptor
> ---------------------------------------------------------------------------------
>
> Key: CHUKWA-332
> URL: https://issues.apache.org/jira/browse/CHUKWA-332
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Components: data collection
> Affects Versions: 0.3.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Fix For: 0.3.0
>
>
> CharFileTailingAdaptorUTF8NewLineEscaped is currently taking Starting offset of the file, and current streaming offset of the file. The original idea of using starting offset as a unique sequence tracker to track the unique record for the life time of the stream. However that idea didn't pan out. Hence, we should remove the first offset to avoid confusion. The current streaming offset is sufficient to start tailing file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-332) Remove first offset tracker from
CharFileTailingAdaptorUTF8NewLineEscaped adaptor
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-332:
------------------------------
Fix Version/s: (was: 0.3.0)
> Remove first offset tracker from CharFileTailingAdaptorUTF8NewLineEscaped adaptor
> ---------------------------------------------------------------------------------
>
> Key: CHUKWA-332
> URL: https://issues.apache.org/jira/browse/CHUKWA-332
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Components: data collection
> Affects Versions: 0.3.0
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
>
> CharFileTailingAdaptorUTF8NewLineEscaped is currently taking Starting offset of the file, and current streaming offset of the file. The original idea of using starting offset as a unique sequence tracker to track the unique record for the life time of the stream. However that idea didn't pan out. Hence, we should remove the first offset to avoid confusion. The current streaming offset is sufficient to start tailing file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.