You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Russell Jurney <ru...@gmail.com> on 2012/02/17 04:57:09 UTC
State of Pig / EMR & S3
I am unable to cp, ls or do anything against s3:// or s3n:// data. I am
unable to LOAD it either. In their Pig, or mine.
Is anyone doing this? How?
--
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com
Re: State of Pig / EMR & S3
Posted by Russell Jurney <ru...@gmail.com>.
Thanks, looks like my issues are Avro related.
Russell Jurney
twitter.com/rjurney
russell.jurney@gmail.com
datasyndrome.com
On Feb 17, 2012, at 9:45 AM, Grig Gheorghiu <gr...@gmail.com> wrote:
> If you're talking about Elastic MapReduce, I am able to LOAD from and
> STORE into S3. Something like this works for me:
>
> concept = LOAD 's3://mybucket/test_files/geowordnet/concept.csv.gz'
> USING PigStorage(',') as (
> con_id: int,
> name: chararray,
> gloss:chararray,
> lang: chararray,
> provenance: chararray);
>
> STORE R12 INTO 's3://mybucket/test_files/geowordnet/pigoutput/R12_$TIMESTAMP';
>
>
> On Thu, Feb 16, 2012 at 7:57 PM, Russell Jurney
> <ru...@gmail.com> wrote:
>> I am unable to cp, ls or do anything against s3:// or s3n:// data. I am
>> unable to LOAD it either. In their Pig, or mine.
>>
>> Is anyone doing this? How?
>>
>> --
>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com
Re: State of Pig / EMR & S3
Posted by Grig Gheorghiu <gr...@gmail.com>.
If you're talking about Elastic MapReduce, I am able to LOAD from and
STORE into S3. Something like this works for me:
concept = LOAD 's3://mybucket/test_files/geowordnet/concept.csv.gz'
USING PigStorage(',') as (
con_id: int,
name: chararray,
gloss:chararray,
lang: chararray,
provenance: chararray);
STORE R12 INTO 's3://mybucket/test_files/geowordnet/pigoutput/R12_$TIMESTAMP';
On Thu, Feb 16, 2012 at 7:57 PM, Russell Jurney
<ru...@gmail.com> wrote:
> I am unable to cp, ls or do anything against s3:// or s3n:// data. I am
> unable to LOAD it either. In their Pig, or mine.
>
> Is anyone doing this? How?
>
> --
> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com