You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Richa Khandelwal <ri...@gmail.com> on 2009/03/05 17:13:06 UTC
Hadoop AMI for EC2
Hi All,
Is there an existing Hadoop AMI for EC2 which had Hadaoop setup on it?
Thanks,
Richa Khandelwal
University Of California,
Santa Cruz.
Ph:425-241-7763
Re: Hadoop AMI for EC2
Posted by Richa Khandelwal <ri...@gmail.com>.
Hi All,
I am trying trying to log map reduce jobs in HADOOP_LOG_DIR by setting its
value in hadoop-env.sh. But the directory has no log records when the job
finishes running. I am adding JobConf.setProfileEnabled(true) in my job. Can
anyone point out how to log in hadoop?
Thanks,
Richa
On Thu, Mar 5, 2009 at 8:20 AM, Richa Khandelwal <ri...@gmail.com> wrote:
> Thats pretty cool. Thanks
>
>
> On Thu, Mar 5, 2009 at 8:17 AM, tim robertson <ti...@gmail.com>wrote:
>
>> Yeps,
>>
>> A good starting read: http://wiki.apache.org/hadoop/AmazonEC2
>>
>> These are the AMIs:
>>
>> $ ec2-describe-images -a | grep hadoop
>> IMAGE ami-245db94d cloudbase-1.1-hadoop-fc64/image.manifest.xml
>> 247610401714 available public x86_64 machine
>> IMAGE ami-791ffb10
>> cloudbase-hadoop-fc64/cloudbase-hadoop-fc64.manifest.xml
>> 247610401714 available public x86_64 machine
>> IMAGE ami-f73adf9e cs345-hadoop-EC2-0.15.3/hadoop-0.15.3.manifest.xml
>> 825431212034 available public i386 machine
>> IMAGE ami-c55db8ac fedora8-hypertable-hadoop-kfs/image.manifest.xml
>> 291354417104 available public x86_64 machine
>> aki-b51cf9dc ari-b31cf9da
>> IMAGE ami-ce6b8fa7 hachero-hadoop/hadoop-0.19.0-i386.manifest.xml
>> 118946012109 available public i386 machine
>> aki-a71cf9ce ari-a51cf9cc
>> IMAGE ami-dd48acb4 hachero-hadoop/hadoop-0.19.0-x86_64.manifest.xml
>> 118946012109 available public x86_64 machine
>> aki-b51cf9dc ari-b31cf9da
>> IMAGE ami-ee53b687 hadoop-ec2-images/hadoop-0.17.0-i386.manifest.xml
>> 111560892610 available public i386 machine
>> aki-a71cf9ce ari-a51cf9cc
>> IMAGE ami-f853b691
>> hadoop-ec2-images/hadoop-0.17.0-x86_64.manifest.xml 111560892610
>> available public x86_64 machine aki-b51cf9dc
>> ari-b31cf9da
>> IMAGE ami-65987c0c hadoop-images/hadoop-0.17.1-i386.manifest.xml
>> 914733919441 available public i386 machine aki-a71cf9ce
>> ari-a51cf9cc
>> IMAGE ami-4b987c22 hadoop-images/hadoop-0.17.1-x86_64.manifest.xml
>> 914733919441 available public x86_64 machine aki-b51cf9dc
>> ari-b31cf9da
>> IMAGE ami-b0fe1ad9 hadoop-images/hadoop-0.18.0-i386.manifest.xml
>> 914733919441 available public i386 machine aki-a71cf9ce
>> ari-a51cf9cc
>> IMAGE ami-90fe1af9 hadoop-images/hadoop-0.18.0-x86_64.manifest.xml
>> 914733919441 available public x86_64 machine aki-b51cf9dc
>> ari-b31cf9da
>> IMAGE ami-ea36d283 hadoop-images/hadoop-0.18.1-i386.manifest.xml
>> 914733919441 available public i386 machine aki-a71cf9ce
>> ari-a51cf9cc
>> IMAGE ami-fe37d397 hadoop-images/hadoop-0.18.1-x86_64.manifest.xml
>> 914733919441 available public x86_64 machine aki-b51cf9dc
>> ari-b31cf9da
>> IMAGE ami-fa6a8e93 hadoop-images/hadoop-0.19.0-i386.manifest.xml
>> 914733919441 available public i386 machine aki-a71cf9ce
>> ari-a51cf9cc
>> IMAGE ami-cd6a8ea4 hadoop-images/hadoop-0.19.0-x86_64.manifest.xml
>> 914733919441 available public x86_64 machine aki-b51cf9dc
>> ari-b31cf9da
>> IMAGE ami-15e80f7c
>> hadoop-images/hadoop-base-20090210-i386.manifest.xml 914733919441
>> available public i386 machine aki-a71cf9ce
>> ari-a51cf9cc
>> IMAGE ami-1ee80f77
>> hadoop-images/hadoop-base-20090210-x86_64.manifest.xml 914733919441
>> available public x86_64 machine aki-b51cf9dc
>> ari-b31cf9da
>> IMAGE ami-4de30724
>> hbase-ami/hbase-0.2.0-hadoop-0.17.1-i386.manifest.xml 834125115996
>> available public i386 machine aki-a71cf9ce
>> ari-a51cf9cc
>> IMAGE ami-fe7c9997 radlab-hadoop-4-large/image.manifest.xml
>> 117716615155 available public x86_64 machine
>> IMAGE ami-7f7f9a16 radlab-hadoop-4/image.manifest.xml
>> 117716615155 available public i386 machine
>> $
>>
>> Cheers,
>>
>> Tim
>>
>>
>>
>> On Thu, Mar 5, 2009 at 5:13 PM, Richa Khandelwal <ri...@gmail.com>
>> wrote:
>> > Hi All,
>> > Is there an existing Hadoop AMI for EC2 which had Hadaoop setup on it?
>> >
>> > Thanks,
>> > Richa Khandelwal
>> >
>> >
>> > University Of California,
>> > Santa Cruz.
>> > Ph:425-241-7763
>> >
>>
>
>
>
> --
> Richa Khandelwal
>
>
> University Of California,
> Santa Cruz.
> Ph:425-241-7763
>
--
Richa Khandelwal
University Of California,
Santa Cruz.
Ph:425-241-7763
Re: Hadoop AMI for EC2
Posted by Richa Khandelwal <ri...@gmail.com>.
Thats pretty cool. Thanks
On Thu, Mar 5, 2009 at 8:17 AM, tim robertson <ti...@gmail.com>wrote:
> Yeps,
>
> A good starting read: http://wiki.apache.org/hadoop/AmazonEC2
>
> These are the AMIs:
>
> $ ec2-describe-images -a | grep hadoop
> IMAGE ami-245db94d cloudbase-1.1-hadoop-fc64/image.manifest.xml
> 247610401714 available public x86_64 machine
> IMAGE ami-791ffb10
> cloudbase-hadoop-fc64/cloudbase-hadoop-fc64.manifest.xml
> 247610401714 available public x86_64 machine
> IMAGE ami-f73adf9e cs345-hadoop-EC2-0.15.3/hadoop-0.15.3.manifest.xml
> 825431212034 available public i386 machine
> IMAGE ami-c55db8ac fedora8-hypertable-hadoop-kfs/image.manifest.xml
> 291354417104 available public x86_64 machine
> aki-b51cf9dc ari-b31cf9da
> IMAGE ami-ce6b8fa7 hachero-hadoop/hadoop-0.19.0-i386.manifest.xml
> 118946012109 available public i386 machine
> aki-a71cf9ce ari-a51cf9cc
> IMAGE ami-dd48acb4 hachero-hadoop/hadoop-0.19.0-x86_64.manifest.xml
> 118946012109 available public x86_64 machine
> aki-b51cf9dc ari-b31cf9da
> IMAGE ami-ee53b687 hadoop-ec2-images/hadoop-0.17.0-i386.manifest.xml
> 111560892610 available public i386 machine
> aki-a71cf9ce ari-a51cf9cc
> IMAGE ami-f853b691 hadoop-ec2-images/hadoop-0.17.0-x86_64.manifest.xml
> 111560892610 available public x86_64 machine
> aki-b51cf9dc ari-b31cf9da
> IMAGE ami-65987c0c hadoop-images/hadoop-0.17.1-i386.manifest.xml
> 914733919441 available public i386 machine aki-a71cf9ce
> ari-a51cf9cc
> IMAGE ami-4b987c22 hadoop-images/hadoop-0.17.1-x86_64.manifest.xml
> 914733919441 available public x86_64 machine aki-b51cf9dc
> ari-b31cf9da
> IMAGE ami-b0fe1ad9 hadoop-images/hadoop-0.18.0-i386.manifest.xml
> 914733919441 available public i386 machine aki-a71cf9ce
> ari-a51cf9cc
> IMAGE ami-90fe1af9 hadoop-images/hadoop-0.18.0-x86_64.manifest.xml
> 914733919441 available public x86_64 machine aki-b51cf9dc
> ari-b31cf9da
> IMAGE ami-ea36d283 hadoop-images/hadoop-0.18.1-i386.manifest.xml
> 914733919441 available public i386 machine aki-a71cf9ce
> ari-a51cf9cc
> IMAGE ami-fe37d397 hadoop-images/hadoop-0.18.1-x86_64.manifest.xml
> 914733919441 available public x86_64 machine aki-b51cf9dc
> ari-b31cf9da
> IMAGE ami-fa6a8e93 hadoop-images/hadoop-0.19.0-i386.manifest.xml
> 914733919441 available public i386 machine aki-a71cf9ce
> ari-a51cf9cc
> IMAGE ami-cd6a8ea4 hadoop-images/hadoop-0.19.0-x86_64.manifest.xml
> 914733919441 available public x86_64 machine aki-b51cf9dc
> ari-b31cf9da
> IMAGE ami-15e80f7c
> hadoop-images/hadoop-base-20090210-i386.manifest.xml 914733919441
> available public i386 machine aki-a71cf9ce
> ari-a51cf9cc
> IMAGE ami-1ee80f77
> hadoop-images/hadoop-base-20090210-x86_64.manifest.xml 914733919441
> available public x86_64 machine aki-b51cf9dc
> ari-b31cf9da
> IMAGE ami-4de30724
> hbase-ami/hbase-0.2.0-hadoop-0.17.1-i386.manifest.xml 834125115996
> available public i386 machine aki-a71cf9ce
> ari-a51cf9cc
> IMAGE ami-fe7c9997 radlab-hadoop-4-large/image.manifest.xml
> 117716615155 available public x86_64 machine
> IMAGE ami-7f7f9a16 radlab-hadoop-4/image.manifest.xml
> 117716615155 available public i386 machine
> $
>
> Cheers,
>
> Tim
>
>
>
> On Thu, Mar 5, 2009 at 5:13 PM, Richa Khandelwal <ri...@gmail.com>
> wrote:
> > Hi All,
> > Is there an existing Hadoop AMI for EC2 which had Hadaoop setup on it?
> >
> > Thanks,
> > Richa Khandelwal
> >
> >
> > University Of California,
> > Santa Cruz.
> > Ph:425-241-7763
> >
>
--
Richa Khandelwal
University Of California,
Santa Cruz.
Ph:425-241-7763
Re: Hadoop AMI for EC2
Posted by tim robertson <ti...@gmail.com>.
Yeps,
A good starting read: http://wiki.apache.org/hadoop/AmazonEC2
These are the AMIs:
$ ec2-describe-images -a | grep hadoop
IMAGE ami-245db94d cloudbase-1.1-hadoop-fc64/image.manifest.xml 247610401714 available public x86_64 machine
IMAGE ami-791ffb10 cloudbase-hadoop-fc64/cloudbase-hadoop-fc64.manifest.xml 247610401714 available public x86_64 machine
IMAGE ami-f73adf9e cs345-hadoop-EC2-0.15.3/hadoop-0.15.3.manifest.xml 825431212034 available public i386 machine
IMAGE ami-c55db8ac fedora8-hypertable-hadoop-kfs/image.manifest.xml 291354417104 available public x86_64 machine aki-b51cf9dc ari-b31cf9da
IMAGE ami-ce6b8fa7 hachero-hadoop/hadoop-0.19.0-i386.manifest.xml 118946012109 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-dd48acb4 hachero-hadoop/hadoop-0.19.0-x86_64.manifest.xml 118946012109 available public x86_64 machine aki-b51cf9dc ari-b31cf9da
IMAGE ami-ee53b687 hadoop-ec2-images/hadoop-0.17.0-i386.manifest.xml 111560892610 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-f853b691 hadoop-ec2-images/hadoop-0.17.0-x86_64.manifest.xml 111560892610 available public x86_64 machine aki-b51cf9dc ari-b31cf9da
IMAGE ami-65987c0c hadoop-images/hadoop-0.17.1-i386.manifest.xml 914733919441 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-4b987c22 hadoop-images/hadoop-0.17.1-x86_64.manifest.xml 914733919441 available public x86_64 machine aki-b51cf9dc ari-b31cf9da
IMAGE ami-b0fe1ad9 hadoop-images/hadoop-0.18.0-i386.manifest.xml 914733919441 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-90fe1af9 hadoop-images/hadoop-0.18.0-x86_64.manifest.xml 914733919441 available public x86_64 machine aki-b51cf9dc ari-b31cf9da
IMAGE ami-ea36d283 hadoop-images/hadoop-0.18.1-i386.manifest.xml 914733919441 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-fe37d397 hadoop-images/hadoop-0.18.1-x86_64.manifest.xml 914733919441 available public x86_64 machine aki-b51cf9dc ari-b31cf9da
IMAGE ami-fa6a8e93 hadoop-images/hadoop-0.19.0-i386.manifest.xml 914733919441 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-cd6a8ea4 hadoop-images/hadoop-0.19.0-x86_64.manifest.xml 914733919441 available public x86_64 machine aki-b51cf9dc ari-b31cf9da
IMAGE ami-15e80f7c hadoop-images/hadoop-base-20090210-i386.manifest.xml 914733919441 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-1ee80f77 hadoop-images/hadoop-base-20090210-x86_64.manifest.xml 914733919441 available public x86_64 machine aki-b51cf9dc ari-b31cf9da
IMAGE ami-4de30724 hbase-ami/hbase-0.2.0-hadoop-0.17.1-i386.manifest.xml 834125115996 available public i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-fe7c9997 radlab-hadoop-4-large/image.manifest.xml 117716615155 available public x86_64 machine
IMAGE ami-7f7f9a16 radlab-hadoop-4/image.manifest.xml 117716615155 available public i386 machine
$
Cheers,
Tim
On Thu, Mar 5, 2009 at 5:13 PM, Richa Khandelwal <ri...@gmail.com> wrote:
> Hi All,
> Is there an existing Hadoop AMI for EC2 which had Hadaoop setup on it?
>
> Thanks,
> Richa Khandelwal
>
>
> University Of California,
> Santa Cruz.
> Ph:425-241-7763
>
Re: Hadoop AMI for EC2
Posted by Tom White <to...@cloudera.com>.
Hi Richa,
Yes there is. Please see http://wiki.apache.org/hadoop/AmazonEC2.
Tom
On Thu, Mar 5, 2009 at 4:13 PM, Richa Khandelwal <ri...@gmail.com> wrote:
> Hi All,
> Is there an existing Hadoop AMI for EC2 which had Hadaoop setup on it?
>
> Thanks,
> Richa Khandelwal
>
>
> University Of California,
> Santa Cruz.
> Ph:425-241-7763
>