header record for each created hdfs file

[Raymond Ng <ra...@gmail.com>]

Hi

is it possible to include a header record for each hdfs file that the hdfs
sink rolls over?

I have a large csv file to put into hdfs but I'd like to have the csv
columns written to the top of each of the rolled over file

thanks

-- 
Rgds
Ray

Re: header record for each created hdfs file

[Mike Percy <mp...@apache.org>]

Hi Ray,
Have you taken a look @ the EventSerializer interface? It lets you write a
plugin to customize a header or trailer on a per-file basis, in addition to
arbitrary serialization of the Flume Event objects.

http://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/serialization/EventSerializer.html
http://flume.apache.org/FlumeUserGuide.html#hdfs-sink

Regards,
Mike


On Mon, Sep 24, 2012 at 9:05 AM, Raymond Ng <ra...@gmail.com> wrote:

> Hi
>
> is it possible to include a header record for each hdfs file that the hdfs
> sink rolls over?
>
> I have a large csv file to put into hdfs but I'd like to have the csv
> columns written to the top of each of the rolled over file
>
> thanks
>
> --
> Rgds
> Ray
>