You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flume.apache.org by Hari Shreedharan <hs...@cloudera.com> on 2014/12/04 01:19:25 UTC

Re: Loading Directories into HDFS

Flume deals with events, not files. Spooling Dir source can read events from files in a directory on local disk and send it to HDFS, but the events from one file might end up in different files based on configuration, routing etc. Also, Spool Dir source currently can’t handle sub-directories, so any sub-directories are ignored.

Thanks,
Hari

On Sun, Nov 30, 2014 at 8:30 PM, Sai Pavan Gadde <id...@gmail.com>
wrote:

> Hi,
> I would like to load a DIR that contains sub dir and a list of files using
> flume API or java API
> Please could you help me in this regard. I would like to go for coding of
> how to ingest/load dir that contains sub dir's and files. I have an idea
> about files loading into hdfs using flume api/java api
> Thanks in Advance
> -- 
> *THANKS & REGARDS,*
> G.SAI PAVAN,
> CCDH4 CERTIFIED,
> *www.bigdatatrendz.com <http://www.bigdatatrendz.com>*
> linkedin profile <http://in.linkedin.com/pub/gadde-sai-pavan/38/44b/453/>
> HYDERABAD.