[jira] Updated: (CHUKWA-462) Store the cluster in the key for performance and easier customization on mappers

["Guille -bisho- (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CHUKWA-462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Guille -bisho- updated CHUKWA-462:
----------------------------------

    Attachment: cluster_in_ChukwaRecordKey.v3.diff

Proposed patch

> Store the cluster in the key for performance and easier customization on mappers
> --------------------------------------------------------------------------------
>
>                 Key: CHUKWA-462
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-462
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: Data Processors
>            Reporter: Guille -bisho-
>         Attachments: cluster_in_ChukwaRecordKey.v3.diff
>
>
> Right now the chukwa framework is storing the destination cluster as a tag in the Chunk. Then the tags are copied to the ChukwaRecord, and before storing it, it's parsed with a regular expression from each record.
> - It's slow to apply a preg to each record
> - It's harder to modify the destination cluster from the mapper, you have to tweak the tags field.
> - Takes unneeded space on records storing the cluster on each of them.
> The proposed path:
> - Extracts the cluster from chunk tags just once per chunk, much faster.
> - Stores the cluster in the key, so it's easy to recover.
> - It's easy to tweak from the mapper. Just alter it with key.setClusterName(String clusterName)
> - Strips the cluster from the tags field of the resulting chukwa records. If the tags field is empty, completely skips setting the tags field in the record.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.