You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Flinkster <fl...@gmail.com> on 2012/03/12 05:53:36 UTC

Python logs to Flume?

Hi there,

What's the recommended way to log to Flume from python? Flume
avroSource? I don't really use python SysLogHandler.

I'm sure there are many users logging from python to Flume. Could you
please share your experiences & best practises?

Thanks,

-ari

Re: Python logs to Flume?

Posted by alo alt <wg...@googlemail.com>.
Hi Ari,

in the source code you'll find a thrift api (https://github.com/cloudera/flume/blob/master/src/thrift/flume.thrift), you could use that like:

from thrift.transport import TSocket
from thrift.transport import THttpClient
from thrift.protocol import TBinaryProtocol
....

Or implement a own handler and use syslog as protocol. 

best,
 Alex 


--
Alexander Lorenz
http://mapredit.blogspot.com

On Mar 20, 2012, at 7:59 AM, Flinkster wrote:

> Alex,
> 
> Thanks for the advice. I'm not familiar with python, do I need to
> implement a new python LogHandler for avro?
> 
> -ari
> 
> On Mon, Mar 12, 2012 at 12:21 AM, alo alt <wg...@googlemail.com> wrote:
>> Hi,
>> 
>> Avro is the best way. Or the thrift python interface generated from src/thrift/flume.thrift.
>> 
>> best,
>>  Alex
>> 
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>> 
>> On Mar 12, 2012, at 5:53 AM, Flinkster wrote:
>> 
>>> Hi there,
>>> 
>>> What's the recommended way to log to Flume from python? Flume
>>> avroSource? I don't really use python SysLogHandler.
>>> 
>>> I'm sure there are many users logging from python to Flume. Could you
>>> please share your experiences & best practises?
>>> 
>>> Thanks,
>>> 
>>> -ari
>>

Re: Python logs to Flume?

Posted by Flinkster <fl...@gmail.com>.
Alex,

Thanks for the advice. I'm not familiar with python, do I need to
implement a new python LogHandler for avro?

-ari

On Mon, Mar 12, 2012 at 12:21 AM, alo alt <wg...@googlemail.com> wrote:
> Hi,
>
> Avro is the best way. Or the thrift python interface generated from src/thrift/flume.thrift.
>
> best,
>  Alex
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Mar 12, 2012, at 5:53 AM, Flinkster wrote:
>
>> Hi there,
>>
>> What's the recommended way to log to Flume from python? Flume
>> avroSource? I don't really use python SysLogHandler.
>>
>> I'm sure there are many users logging from python to Flume. Could you
>> please share your experiences & best practises?
>>
>> Thanks,
>>
>> -ari
>

Re: Python logs to Flume?

Posted by alo alt <wg...@googlemail.com>.
Hi,

Avro is the best way. Or the thrift python interface generated from src/thrift/flume.thrift.

best,
 Alex 

--
Alexander Lorenz
http://mapredit.blogspot.com

On Mar 12, 2012, at 5:53 AM, Flinkster wrote:

> Hi there,
> 
> What's the recommended way to log to Flume from python? Flume
> avroSource? I don't really use python SysLogHandler.
> 
> I'm sure there are many users logging from python to Flume. Could you
> please share your experiences & best practises?
> 
> Thanks,
> 
> -ari