You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Bojan Kostić <bl...@gmail.com> on 2013/04/08 10:55:51 UTC

Read tmp files from hadoop

Hallo, i have setup flume to write my logs on hadoop. But flume creates tmp
files which i can cat and read using hdfs dfs -cat.
But when i try to read that data i only get first line.

Is there a way to read this tmp file?

Re: Read tmp files from hadoop

Posted by Prashant Kommireddi <pr...@gmail.com>.

Bojan,

Do these files start with a period "." or underscore "_" ? Pig/hadoop jobs
ignore these files by default. Is your question about how to read these in
a MR job through a pig script?

On Mon, Apr 8, 2013 at 1:55 AM, Bojan Kostić <bl...@gmail.com> wrote:

> Hallo, i have setup flume to write my logs on hadoop. But flume creates tmp
> files which i can cat and read using hdfs dfs -cat.
> But when i try to read that data i only get first line.
>
> Is there a way to read this tmp file?
>