You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Yulia Tolskaya <yu...@magnetic.is> on 2012/01/13 22:35:18 UTC

STORING each relation in it's own file

Hello,
I am trying to store each line in a relation in it's own file, and avoid having one huge output file.
I cannot seem to find a good way to do this. I would basically like to have something equivalent to a STORE in a FOREACH. What is the best way to do this?


Yulia

Re: STORING each relation in it's own file

Posted by Stan Rosenberg <sr...@proclivitysystems.com>.

Hi Yulia,

One way to accomplish this is by writing your own StoreFunc.  Take a
look at org.apache.pig.piggybank.storage.MultiStorage.  You'd need to
create your own output format and possibly a record writer.

stan

On Fri, Jan 13, 2012 at 4:35 PM, Yulia Tolskaya <yu...@magnetic.is> wrote:
> Hello,
> I am trying to store each line in a relation in it's own file, and avoid having one huge output file.
> I cannot seem to find a good way to do this. I would basically like to have something equivalent to a STORE in a FOREACH. What is the best way to do this?
>
>
> Yulia