How to load zip file into hdfs sink using flume-ng

[Priyanka Jain <ja...@gmail.com>]

Hi,,,
I want to know is it possible to load the zip file from remote location to
the hdfs sink in flume-ng-1.2

Thanks in advance,
Priyanka

Re: How to load zip file into hdfs sink using flume-ng

[Mike Percy <mp...@apache.org>]

Actually it might be tricky to use the directory spooling source to read a
compressed archive. It's possible, but you would definitely need to write
your own deserializer.

Flume is an event-oriented streaming system, it's not really optimized to
be a plain file transfer mechanism like FTP.

Regards,
Mike


On Fri, Feb 8, 2013 at 9:03 AM, Nitin Pawar <ni...@gmail.com> wrote:

> priyanka,
> you can do that flume-ng-1.3 using directory spooling
>
> just be aware what kind of channel you use
>
>
> On Fri, Feb 8, 2013 at 10:06 PM, Priyanka Jain <jainpriyanka1110@gmail.com
> > wrote:
>
>> Hi,,,
>> I want to know is it possible to load the zip file from remote location
>> to the hdfs sink in flume-ng-1.2
>>
>> Thanks in advance,
>> Priyanka
>>
>
>
>
> --
> Nitin Pawar
>

Re: How to load zip file into hdfs sink using flume-ng

[Nitin Pawar <ni...@gmail.com>]

priyanka,
you can do that flume-ng-1.3 using directory spooling

just be aware what kind of channel you use


On Fri, Feb 8, 2013 at 10:06 PM, Priyanka Jain
<ja...@gmail.com>wrote:

> Hi,,,
> I want to know is it possible to load the zip file from remote location to
> the hdfs sink in flume-ng-1.2
>
> Thanks in advance,
> Priyanka
>



-- 
Nitin Pawar