You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by charles du <ta...@gmail.com> on 2008/08/06 23:56:47 UTC
Load list of records with missing fields
Hi:
I have a list of records stored in a file. Most of records have 10 fields,
but some of records have less fields.
I would like to filter out these records with less fields. How could I do
it? Is there a way to get number of fields in a given record, like '$#' in
shell program so I can do
> A = LOAD "my file" ;
> A_CLEAN = FILTER A BY $# == '10';
Thanks.
--
tp
Re: Load list of records with missing fields
Posted by Alan Gates <ga...@yahoo-inc.com>.
A = LOAD 'myfile';
A_CLEAN = FILTER A BY ARITY(*) == '10';
Alan.
charles du wrote:
> Hi:
>
> I have a list of records stored in a file. Most of records have 10 fields,
> but some of records have less fields.
>
> I would like to filter out these records with less fields. How could I do
> it? Is there a way to get number of fields in a given record, like '$#' in
> shell program so I can do
>
>
>> A = LOAD "my file" ;
>> A_CLEAN = FILTER A BY $# == '10';
>>
>
> Thanks.
>
>
Re: Load list of records with missing fields
Posted by Tanton Gibbs <ta...@gmail.com>.
I believe you want the ARITY function. See the pig builtin functions
here: http://wiki.apache.org/pig/PigBuiltins
On Wed, Aug 6, 2008 at 4:56 PM, charles du <ta...@gmail.com> wrote:
> Hi:
>
> I have a list of records stored in a file. Most of records have 10 fields,
> but some of records have less fields.
>
> I would like to filter out these records with less fields. How could I do
> it? Is there a way to get number of fields in a given record, like '$#' in
> shell program so I can do
>
>> A = LOAD "my file" ;
>> A_CLEAN = FILTER A BY $# == '10';
>
> Thanks.
>
> --
> tp
>
RE: Load list of records with missing fields
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
You can use ARITY function:
A_CLEAB = filter A by ARITY(*)==10;
> -----Original Message-----
> From: charles du [mailto:taiping.du@gmail.com]
> Sent: Wednesday, August 06, 2008 2:57 PM
> To: pig-user@incubator.apache.org
> Subject: Load list of records with missing fields
>
> Hi:
>
> I have a list of records stored in a file. Most of records
> have 10 fields, but some of records have less fields.
>
> I would like to filter out these records with less fields.
> How could I do it? Is there a way to get number of fields in
> a given record, like '$#' in shell program so I can do
>
> > A = LOAD "my file" ;
> > A_CLEAN = FILTER A BY $# == '10';
>
> Thanks.
>
> --
> tp
>