Hbase Serializers

[Young Kim <yk...@crunchyroll.com>]

Hey folks,

Does anyone happen to have some sort of guide or information on building and using custom serializers for the Hbase sink?

Thanks,
Young Kim 

Re: Hbase Sink

[Mohammad Tariq <do...@gmail.com>]

Hi George,

         We need to add the payload column as well.Add the following
line in your config file :
hbase-agent.sinks.sink1.serializer.payloadColumn = col_name

Regards,
    Mohammad Tariq


On Tue, Jul 17, 2012 at 8:23 PM, George Forman
<ge...@hotmail.com> wrote:
> Hi all,
>
> Newbie here...I am trying to get flume ng to do a simple hbase sink.
> I have downloaded/installed flume-ng and configured it based on
> info found on the net. I don't see flume connecting to hbase.
> I am wondering what I could be missing?  - Thanks
>
> Here is my configuration:
>
> hbase-agent.sources = tail
> hbase-agent.sinks = sink1
> hbase-agent.channels = ch1
>
> hbase-agent.sources.tail.type = exec
> hbase-agent.sources.tail.bind = pseudo
> hbase-agent.sources.tail.command = tail -F /tmp/test-flume.ens
> hbase-agent.sources.tail.channels = ch1
>
> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
> hbase-agent.sinks.sink1.channel = ch1
> hbase-agent.sinks.sink1.table = ens
> hbase-agent.sinks.sink1.columnFamily = raw
> hbase-agent.sinks.sink1.column = event
> hbase-agent.sinks.sink1.serializer =
> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
> hbase-agent.channels.ch1.type = memory
>
>

Hbase Sink

[George Forman <ge...@hotmail.com>]

Hi all,

Newbie here...I am trying to get flume ng to do a simple hbase sink. I have downloaded/installed flume-ng and configured it based oninfo found on the net. I don't see flume connecting to hbase.I am wondering what I could be missing?  - Thanks
Here is my configuration:
hbase-agent.sources = tailhbase-agent.sinks = sink1hbase-agent.channels = ch1
hbase-agent.sources.tail.type = exechbase-agent.sources.tail.bind = pseudohbase-agent.sources.tail.command = tail -F /tmp/test-flume.enshbase-agent.sources.tail.channels = ch1
hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSinkhbase-agent.sinks.sink1.channel = ch1hbase-agent.sinks.sink1.table = enshbase-agent.sinks.sink1.columnFamily = rawhbase-agent.sinks.sink1.column = eventhbase-agent.sinks.sink1.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializerhbase-agent.channels.ch1.type = memory

 		 	   		  

Old flume 0.9.4 source code

[George Forman <ge...@hotmail.com>]

Hi all,

I have searched around but I have failed to find a copy of Apache Flume flume-0.9.4+25.48 version (ie.  flume-0.9.4-cdh4.0.0.tar.gz ).
Does anyone have this still lying around?
Thanks 		 	   		  

Re: Hbase Serializers

[Brock Noland <br...@cloudera.com>]

I am not aware of a guide, but the code is available. You need to implement
this interface:

https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java

example:

https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java



On Fri, Jul 6, 2012 at 6:43 PM, Young Kim <yk...@crunchyroll.com> wrote:

>  Hey folks,
>
> Does anyone happen to have some sort of guide or information on building
> and using custom serializers for the Hbase sink?
>
> Thanks,
> Young Kim
>
>


-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/