You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Mohammad Tariq <do...@gmail.com> on 2011/11/13 18:48:08 UTC
Collecting xml files using flume
I would like to know whether it is possible to collect all the xml
files from a directory continuously as they are coming and dump these
file onto the HDFS.
Many thanks in advance.
Regards,
Mohammad Tariq
Re: Collecting xml files using flume
Posted by Mohammad Tariq <do...@gmail.com>.
Thank you so much both of you for the help..:)
Regards,
Mohammad Tariq
On Thu, Nov 17, 2011 at 1:47 PM, Alexander C.H. Lorenz
<wg...@googlemail.com> wrote:
> Hi,
> you can use tailDir(“/path/to/directory/”, “*.xml”)
> regards,
> Alex
>
> On Thu, Nov 17, 2011 at 9:06 AM, Mingjie Lai <mj...@gmail.com> wrote:
>>
>> You can try tailDir which can tail multiple files under one directory.
>>
>> But, do the xml files continue growing? It's a little bit weird for me.
>>
>> -mingjie
>>
>> On 11/13/2011 09:48 AM, Mohammad Tariq wrote:
>>>
>>> I would like to know whether it is possible to collect all the xml
>>> files from a directory continuously as they are coming and dump these
>>> file onto the HDFS.
>>> Many thanks in advance.
>>>
>>>
>>> Regards,
>>> Mohammad Tariq
>>>
>
>
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
> P Think of the environment: please don't print this email unless you really
> need to.
>
>
Re: Collecting xml files using flume
Posted by "Alexander C.H. Lorenz" <wg...@googlemail.com>.
Hi,
you can use tailDir(“/path/to/directory/”, “*.xml”)
regards,
Alex
On Thu, Nov 17, 2011 at 9:06 AM, Mingjie Lai <mj...@gmail.com> wrote:
> You can try tailDir which can tail multiple files under one directory.
>
> But, do the xml files continue growing? It's a little bit weird for me.
>
> -mingjie
>
>
> On 11/13/2011 09:48 AM, Mohammad Tariq wrote:
>
>> I would like to know whether it is possible to collect all the xml
>> files from a directory continuously as they are coming and dump these
>> file onto the HDFS.
>> Many thanks in advance.
>>
>>
>> Regards,
>> Mohammad Tariq
>>
>>
--
Alexander Lorenz
http://mapredit.blogspot.com
*P **Think of the environment: please don't print this email unless you
really need to.*
Re: Collecting xml files using flume
Posted by Mingjie Lai <mj...@gmail.com>.
You can try tailDir which can tail multiple files under one directory.
But, do the xml files continue growing? It's a little bit weird for me.
-mingjie
On 11/13/2011 09:48 AM, Mohammad Tariq wrote:
> I would like to know whether it is possible to collect all the xml
> files from a directory continuously as they are coming and dump these
> file onto the HDFS.
> Many thanks in advance.
>
>
> Regards,
> Mohammad Tariq
>