You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "jaydeep ayachit (JIRA)" <ji...@apache.org> on 2010/09/28 16:38:32 UTC
[jira] Created: (CHUKWA-526) Supporting sequential sequence number
in chukwa agent
Supporting sequential sequence number in chukwa agent
-----------------------------------------------------
Key: CHUKWA-526
URL: https://issues.apache.org/jira/browse/CHUKWA-526
Project: Chukwa
Issue Type: Improvement
Components: data collection
Reporter: jaydeep ayachit
The current sequence number is offset in the data. Suggestion is to accompany sequential number that will be available in ChukwaRecord. The sequence number will help to regenerate log file order whenever needed (many times file does not have timestamp or same timestamp for multiple lines). With sequence number, its also possible to query and manipulate using query language like JAQL
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-526) Supporting sequential sequence
number in chukwa agent
Posted by "jaydeep ayachit (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916018#action_12916018 ]
jaydeep ayachit commented on CHUKWA-526:
----------------------------------------
Hi Ari,
The sequence number is not available in ChukwaRecord. As per my understanding, case when file is rotated, the sequence number would reset. The sequence number (record offsets) are used to extract data from the chunk to pass it to demuxer for processing. As such if timestamp is not available in file (or same timestamp across multiple lines) the ordering cannot be determined.
> Supporting sequential sequence number in chukwa agent
> -----------------------------------------------------
>
> Key: CHUKWA-526
> URL: https://issues.apache.org/jira/browse/CHUKWA-526
> Project: Chukwa
> Issue Type: Improvement
> Components: data collection
> Reporter: jaydeep ayachit
>
> The current sequence number is offset in the data. Suggestion is to accompany sequential number that will be available in ChukwaRecord. The sequence number will help to regenerate log file order whenever needed (many times file does not have timestamp or same timestamp for multiple lines). With sequence number, its also possible to query and manipulate using query language like JAQL
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-526) Supporting sequential sequence
number in chukwa agent
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915846#action_12915846 ]
Ari Rabkin commented on CHUKWA-526:
-----------------------------------
Why would numbering records, instead of bytes, make it any easier to recover log file order? Sorting with the existing numbers should give you the same outcome as sorting with sequential numbers.
> Supporting sequential sequence number in chukwa agent
> -----------------------------------------------------
>
> Key: CHUKWA-526
> URL: https://issues.apache.org/jira/browse/CHUKWA-526
> Project: Chukwa
> Issue Type: Improvement
> Components: data collection
> Reporter: jaydeep ayachit
>
> The current sequence number is offset in the data. Suggestion is to accompany sequential number that will be available in ChukwaRecord. The sequence number will help to regenerate log file order whenever needed (many times file does not have timestamp or same timestamp for multiple lines). With sequence number, its also possible to query and manipulate using query language like JAQL
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.