You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Sandesh Hurli <sa...@gmail.com> on 2014/03/07 06:07:07 UTC
Flume RegexHbaseEventSerializer
Hi
I am using Flume's Hbase sink with RegexHbaseEventSerializer, but when I
run the agent I get no entries in the Hbase table.
It works perfectly with SimpleHbaseEventSerializer though.
Here's my sink configuration:
agent.sinks.loggerSink.type = org.apache.flume.sink.hbase.HBaseSink
agent.sinks.loggerSink.table = demo
agent.sinks.loggerSink.columnFamily = cf
agent.sinks.loggerSink.serializer
=org.apache.flume.sink.hbase.RegexHbaseEventSerializer
agent.sinks.loggerSink.serializer.regex
=(.+)\s(.+)\s(\d+)\s(\S+)\s(\S+)\s(\S+)\s(.+)
agent.sinks.loggerSink.serializer.colNames=Date,Time,PID,Level,Module,ReqID,Message
Entries in my source file look like this:
2014-02-27 15:17:54.409 1297 INFO nova.compute.resource_tracker [-]
Compute_service record updated
2014-01-15 09:23:33.234 1288 DEBUG nova.wsgi [-] Loading app ec2 from
/etc/nova/api-paste.ini
I would appreciate any help.
Sandesh.
Re: Flume RegexHbaseEventSerializer
Posted by Sandesh Hurli <sa...@gmail.com>.
Got this working, had to change the regex pattern to
(.+)\\s(.+)\\s(\\d+)\\s(\\S+)\\s(\\S+)\\s(\\S+)\\s(.+). My bad.
Cheers!
On Fri, Mar 7, 2014 at 10:37 AM, Sandesh Hurli <sa...@gmail.com> wrote:
> Hi
>
> I am using Flume's Hbase sink with RegexHbaseEventSerializer, but when I
> run the agent I get no entries in the Hbase table.
> It works perfectly with SimpleHbaseEventSerializer though.
>
> Here's my sink configuration:
>
> agent.sinks.loggerSink.type = org.apache.flume.sink.hbase.HBaseSink
> agent.sinks.loggerSink.table = demo
> agent.sinks.loggerSink.columnFamily = cf
> agent.sinks.loggerSink.serializer
> =org.apache.flume.sink.hbase.RegexHbaseEventSerializer
> agent.sinks.loggerSink.serializer.regex
> =(.+)\s(.+)\s(\d+)\s(\S+)\s(\S+)\s(\S+)\s(.+)
>
> agent.sinks.loggerSink.serializer.colNames=Date,Time,PID,Level,Module,ReqID,Message
>
> Entries in my source file look like this:
>
> 2014-02-27 15:17:54.409 1297 INFO nova.compute.resource_tracker [-]
> Compute_service record updated
> 2014-01-15 09:23:33.234 1288 DEBUG nova.wsgi [-] Loading app ec2 from
> /etc/nova/api-paste.ini
>
> I would appreciate any help.
>
> Sandesh.
>