You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Yang <te...@gmail.com> on 2013/04/27 08:52:49 UTC

loaded records but gone

I wrote a record loader MyLoader  and used it to load

aa = LOAD 'input_on_hdfs' USING MyLoader() AS (
blah:chararray,
blahblah:chararray
);

bb = FOREACH aa generate *;
store bb into 'somewhere_else';


when I ran this, pig shows that nnnnnn records were read in, but it shows
that 0 records were written,  how is this possible?

but between the above 2 statements, if I add the following, to store it
into the regular pigStorage() format, and then load it out back into aa,
then it works perfectly

STORE aa into '/blahblahba';
aa = LOAD aa FROM '/blahblahba' as (blah:chararray, blahblah:chararray)
;


where is it possibly  going wrong? is there a workaround ---- storing it
temporarily takes a lot of space and time


thanks
Yang