Amazon Elastic Map-Reduce and Pig

[Mathias Fryde <ma...@mimesis-republic.com>]

Hello,
I was wondering if anyone as test Pig on Amazon Elastic Map-Reduce.
Could anyone can give us a feed back of compatibility etc ...

Witch version of Pig is working on Amazon Elastic Map-Reduce ?

thanks in advance.
Mathias

Re: Amazon Elastic Map-Reduce and Pig

[Alan Gates <ga...@yahoo-inc.com>]

Based on a quick look, it doesn't seem pig will run out of the box  
with Amazon's new offering.  They allow you to submit your jar for  
hadoop via a web page or via a command line utility that uses a web  
services API.  Pig does generate a jar to send to hadoop, but it  
doesn't expose that to the user.  So to work with this new service,  
either Amazon would have to accept pig scripts directly, just as it  
accepts haddop jars (which would be ideal), or pig would have to have  
an option where it could create the jar for the user and then not  
execute it.  The user could then submit this jar to Amazon's service  
(this is obviously less ideal).

Alan.

On Apr 3, 2009, at 11:01 AM, Ted Dunning wrote:

> This is not the same as the original question.  Lots of people use  
> hadoop on
> EC2.  Some use Pig.  The question has more to do with whether it is  
> easy to
> use pig on this latest Amazon offering.
>
> On Fri, Apr 3, 2009 at 12:57 AM, zhang jianfeng <zj...@gmail.com>  
> wrote:
>
>> I've run pig on EC2, and it works. And I use the trunk version in  
>> pig.
>>
>> You can refer to this link: http://www.cloudera.com/hadoop-ec2
>>
>> They provide a script to facilitate the hadoop and pig installation  
>> on EC2.
>>
>>
>>
>>
>>
>> On Fri, Apr 3, 2009 at 3:38 PM, Mathias Fryde <
>> mathias.fryde@mimesis-republic.com> wrote:
>>
>>> Hello,
>>> I was wondering if anyone as test Pig on Amazon Elastic Map-Reduce.
>>> Could anyone can give us a feed back of compatibility etc ...
>>>
>>> Witch version of Pig is working on Amazon Elastic Map-Reduce ?
>>>
>>> thanks in advance.
>>> Mathias
>>>
>>
>
>
>
> -- 
> Ted Dunning, CTO
> DeepDyve
>
> 111 West Evelyn Ave. Ste. 202
> Sunnyvale, CA 94086
> www.deepdyve.com
> 858-414-0013 (m)
> 408-773-0220 (fax)

Re: Amazon Elastic Map-Reduce and Pig

[Ted Dunning <te...@gmail.com>]

This is not the same as the original question.  Lots of people use hadoop on
EC2.  Some use Pig.  The question has more to do with whether it is easy to
use pig on this latest Amazon offering.

On Fri, Apr 3, 2009 at 12:57 AM, zhang jianfeng <zj...@gmail.com> wrote:

> I've run pig on EC2, and it works. And I use the trunk version in pig.
>
> You can refer to this link: http://www.cloudera.com/hadoop-ec2
>
> They provide a script to facilitate the hadoop and pig installation on EC2.
>
>
>
>
>
> On Fri, Apr 3, 2009 at 3:38 PM, Mathias Fryde <
> mathias.fryde@mimesis-republic.com> wrote:
>
> > Hello,
> > I was wondering if anyone as test Pig on Amazon Elastic Map-Reduce.
> > Could anyone can give us a feed back of compatibility etc ...
> >
> > Witch version of Pig is working on Amazon Elastic Map-Reduce ?
> >
> > thanks in advance.
> > Mathias
> >
>



-- 
Ted Dunning, CTO
DeepDyve

111 West Evelyn Ave. Ste. 202
Sunnyvale, CA 94086
www.deepdyve.com
858-414-0013 (m)
408-773-0220 (fax)

Re: Amazon Elastic Map-Reduce and Pig

[zhang jianfeng <zj...@gmail.com>]

I've run pig on EC2, and it works. And I use the trunk version in pig.

You can refer to this link: http://www.cloudera.com/hadoop-ec2

They provide a script to facilitate the hadoop and pig installation on EC2.





On Fri, Apr 3, 2009 at 3:38 PM, Mathias Fryde <
mathias.fryde@mimesis-republic.com> wrote:

> Hello,
> I was wondering if anyone as test Pig on Amazon Elastic Map-Reduce.
> Could anyone can give us a feed back of compatibility etc ...
>
> Witch version of Pig is working on Amazon Elastic Map-Reduce ?
>
> thanks in advance.
> Mathias
>