Regarding Spooling Dir

[mahendran m <ma...@hotmail.com>]

Hi ,
Is there any way to move full file (1MB) from spooling DIr into HDFS as single file ? If possible then how can i achieve this  .
Thanks,Mahendran 		 	   		  

Re: Regarding Spooling Dir

[Joey Echeverria <jo...@cloudera.com>]

This is a good use case for a new Apache project that's in the
incubator called Apache NiFi[1]. I wrote a blog post[2] describing how
you can use NiFi to implement this kind of dropbox functionality. If
you decide to give it a shot and you run into issues, you can e-mail
the NiFi community at dev@nifi.incubator.apache.org.

Cheers,

-Joey

[1] http://nifi.incubator.apache.org
[2] http://ingest.tips/2014/12/22/getting-started-with-apache-nifi/

On Wed, Jan 7, 2015 at 3:36 PM, Hari Shreedharan
<hs...@cloudera.com> wrote:
> Flume is optimized to send events, not files. There are ways to do this, but
> that is not what Flume is optimized for.
>
> Thanks,
> Hari
>
>
> On Tue, Jan 6, 2015 at 10:59 PM, mahendran m <ma...@hotmail.com>
> wrote:
>>
>> Hi ,
>>
>> Is there any way to move full file (1MB) from spooling DIr into HDFS as
>> single file ? If possible then how can i achieve this  .
>>
>> Thanks,
>> Mahendran
>
>



-- 
Joey Echeverria

Re: Regarding Spooling Dir

[Hari Shreedharan <hs...@cloudera.com>]

Flume is optimized to send events, not files. There are ways to do this, but that is not what Flume is optimized for.




Thanks, Hari

On Tue, Jan 6, 2015 at 10:59 PM, mahendran m <ma...@hotmail.com>
wrote:

> Hi ,
> Is there any way to move full file (1MB) from spooling DIr into HDFS as single file ? If possible then how can i achieve this  .
> Thanks,Mahendran