Re: Question about scribe source to flume

[Jonathan Hsieh <jo...@cloudera.com>]

[Please subscribe to new flume-user@incubator.apache.org list, bcc
flume-user@cloudera.org, cc flume-user@incubator.apache.org]

You can set your collector's destination to use the scribe.category
attrbiute.

collector1 : collectorSource(35853) |
collectorSink("hdfs://namenode/flume/%{scribe.category}","testOut");

So let's say you have events with category "foo", "bar", and "" (empty)

Data would appear in:

hdfs://namenode/flume/foo/testOut*
hdfs://namenode/flume/bar/testOut*
hdfs://namenode/flume/testOut*

Jon.

On Sun, Jul 17, 2011 at 4:55 PM, Nanhong Ye <na...@gmail.com> wrote:

> I setup a flume agent to listen scribe events..
>
> First config:   scribe: scribe | console..
>
> It shows the scribe.category with log info...
>
> [INFO Sun Jul 17 23:02:22 GMT+00:00 2011] { scribe.category :
> session_detail } xx.xxx.xxx.xx\United States\CA
>
>
> Second Configure:    scribe : scribe | agentSink("localhost",35853) ;
> collector1 : collectorSource(35853) | collectorSink("hdfs://namenode/
> flume/","testOut");
>
>
>
> Then, all log info goes into hdfs://namenode/flume/ .......
>
>
> My question is how I can put the log info into different categories
> based on scribe.category ???   For example,  I have three categories:
> session_detail, session_qos,and session_properties...  I wanna log
> scribe events of different categories into HDFS....
> ( session_detail ->  hdfs://namenode/flume/session_detail,
> session_qos -> hdfs://namenode/flume/session_qos , etc).
>
>
> Thanks,
> regards.
>
>


-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com