Pig Load function with variable tuple length

[sridhar basam <sr...@basam.org>]

Hey,
       I have a file similar to syslog output. It is 1 tuple per line, space
seperated, but the tuple can have variable number of arguments if you use
the standard PigStorage function to load the file.
The first 4 variables are always defined and have a strict format, the rest
of the line i would like to define as a single chararray (including spaces).
Is there anyway for me to do that in pig?

thanks,
             Sridhar

Re: Pig Load function with variable tuple length

[Dmitriy Ryaboy <dv...@gmail.com>]

The simplest thing to do might be to use the simple TextLoader, and do
the parsing of the lines yourself, using either the various built-in
regex extraction functions, or a custom UDF.

On Thu, May 26, 2011 at 9:10 AM, sridhar basam <sr...@basam.org> wrote:
> Hey,
>       I have a file similar to syslog output. It is 1 tuple per line, space
> seperated, but the tuple can have variable number of arguments if you use
> the standard PigStorage function to load the file.
> The first 4 variables are always defined and have a strict format, the rest
> of the line i would like to define as a single chararray (including spaces).
> Is there anyway for me to do that in pig?
>
> thanks,
>             Sridhar
>