You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by zhang jianfeng <zj...@gmail.com> on 2009/08/05 12:19:57 UTC
Can I filter rows in SerDe ?
Hi all,
It seems SerDe read files line by line. And each time I only get one line.
But in my files, some lines actually do not represent records, e.g. some
lines just represent the meta data of records and some lines are empty
lines. How can I filter these rows in SerDe? Or I have to return null and
filter them in QL ?
Thank you .
Jeff Zhang
Re: Can I filter rows in SerDe ?
Posted by Zheng Shao <zs...@gmail.com>.
Currently SerDe has to return one row from one line.
You can either do what you mentioned, or write a new InputFileFormat
which filters out the non-data lines.
Zheng
On Wed, Aug 5, 2009 at 3:19 AM, zhang jianfeng<zj...@gmail.com> wrote:
> Hi all,
>
> It seems SerDe read files line by line. And each time I only get one line.
> But in my files, some lines actually do not represent records, e.g. some
> lines just represent the meta data of records and some lines are empty
> lines. How can I filter these rows in SerDe? Or I have to return null and
> filter them in QL ?
>
>
> Thank you .
>
>
> Jeff Zhang
>
>
>
--
Yours,
Zheng