keep event header when dump into hbase sink?

["Xu (Simon) Chen" <xc...@gmail.com>]

Hi all,

Which hbase sink keeps the event header, and how to configure it to do so?

The SimpleHbaseEventSerializer would certainly discard the header:
https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java

It doesn't look like the RegexHbaseEventSerializer does anything either.

Is there such an option, or do I have to implement my own serializer?

Basically I have a syslog source where useful information is kept in the header:

Event: { headers:{region=flp, timestamp=1357589618000, Severity=5,
host=switch1, Facility=20} body: 45 62 72 61 3A 20 25 4C 49 4E 45 50
52 4F 54 4F Ebra: %LINEPROTO }

Thanks.
-Simon

Re: keep event header when dump into hbase sink?

[Brock Noland <br...@cloudera.com>]

I would subclass HbaseEventSerializer for this purpose.

On Mon, Jan 7, 2013 at 10:33 AM, Xu (Simon) Chen <xc...@gmail.com> wrote:
> Hi all,
>
> Which hbase sink keeps the event header, and how to configure it to do so?
>
> The SimpleHbaseEventSerializer would certainly discard the header:
> https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java
>
> It doesn't look like the RegexHbaseEventSerializer does anything either.
>
> Is there such an option, or do I have to implement my own serializer?
>
> Basically I have a syslog source where useful information is kept in the header:
>
> Event: { headers:{region=flp, timestamp=1357589618000, Severity=5,
> host=switch1, Facility=20} body: 45 62 72 61 3A 20 25 4C 49 4E 45 50
> 52 4F 54 4F Ebra: %LINEPROTO }
>
> Thanks.
> -Simon



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/