S3 source

[Roman Shaposhnik <rv...@apache.org>]

Hi!

I have, what I would imagine, is a pretty common use case -- a bunch of S3
buckets with access logging enabled. S3 is nice in a sense that the logging
for the bucket all goes back to a bucket of your choice, but of course, I really
want this data in HDFS so I can analyze it.

Is there a flume source that can help me here? I couldn't find one.

Thanks,
Roman.

Re: S3 source

[Mubarak Seyed <se...@apple.com>]

There is a related discussion at

http://mail-archives.apache.org/mod_mbox/flume-user/201208.mbox/%3C3D87E13A11A14E238634A857125CB33C@cloudera.com%3E


Thanks,
Mubarak
On Aug 23, 2012, at 5:44 PM, Mike Percy wrote:

> Roman,
> I haven't seen an implementation floating around, but I would love to get support for an S3 Source into Flume!
> 
> Regards,
> Mike
> 
> On Thu, Aug 23, 2012 at 5:06 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> Hi!
> 
> I have, what I would imagine, is a pretty common use case -- a bunch of S3
> buckets with access logging enabled. S3 is nice in a sense that the logging
> for the bucket all goes back to a bucket of your choice, but of course, I really
> want this data in HDFS so I can analyze it.
> 
> Is there a flume source that can help me here? I couldn't find one.
> 
> Thanks,
> Roman.
> 

Re: S3 source

[Mike Percy <mp...@apache.org>]

Roman,
I haven't seen an implementation floating around, but I would love to get
support for an S3 Source into Flume!

Regards,
Mike

On Thu, Aug 23, 2012 at 5:06 PM, Roman Shaposhnik <rv...@apache.org> wrote:

> Hi!
>
> I have, what I would imagine, is a pretty common use case -- a bunch of S3
> buckets with access logging enabled. S3 is nice in a sense that the logging
> for the bucket all goes back to a bucket of your choice, but of course, I
> really
> want this data in HDFS so I can analyze it.
>
> Is there a flume source that can help me here? I couldn't find one.
>
> Thanks,
> Roman.
>