You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Yang <te...@gmail.com> on 2013/01/09 23:37:30 UTC

generate multiple output files?

let's say I have an input dataset, each row has 2 fields, the first field
is a value among 100 possible values. I want to just split the input
dataset into 100 outputs , based on the  value of the first field.

is there a way to do that in pig? I see MultipleOutputs Format in Java API,
but have not found anything similar in PIG

Thanks!
Yang

Re: generate multiple output files?

Posted by Yang <te...@gmail.com>.

very nice, thanks!


On Fri, Jan 11, 2013 at 2:52 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:

> Yang,
> Try MultiStorage:
>
> https://pig.apache.org/docs/r0.8.1/api/org/apache/pig/piggybank/storage/MultiStorage.html
>
>
> On Wed, Jan 9, 2013 at 2:37 PM, Yang <te...@gmail.com> wrote:
>
> > let's say I have an input dataset, each row has 2 fields, the first field
> > is a value among 100 possible values. I want to just split the input
> > dataset into 100 outputs , based on the  value of the first field.
> >
> > is there a way to do that in pig? I see MultipleOutputs Format in Java
> API,
> > but have not found anything similar in PIG
> >
> > Thanks!
> > Yang
> >
>

Re: generate multiple output files?

Posted by Dmitriy Ryaboy <dv...@gmail.com>.

Yang,
Try MultiStorage:
https://pig.apache.org/docs/r0.8.1/api/org/apache/pig/piggybank/storage/MultiStorage.html


On Wed, Jan 9, 2013 at 2:37 PM, Yang <te...@gmail.com> wrote:

> let's say I have an input dataset, each row has 2 fields, the first field
> is a value among 100 possible values. I want to just split the input
> dataset into 100 outputs , based on the  value of the first field.
>
> is there a way to do that in pig? I see MultipleOutputs Format in Java API,
> but have not found anything similar in PIG
>
> Thanks!
> Yang
>