You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Prabhu Hari Dhanapal <dr...@gmail.com> on 2009/11/16 07:28:25 UTC
Any Way to SSH to a remote Hadoop Machine
Hi,
I have been having lot of problems installing hadoop on my system and I
finally gave up.I was just wondering if I could SSH to a remote hadoop
cluster that is open for developers to practice on ? Is there anything
provided by cloudera ? Please let me know.
Thanks in Advance.
--
Hari
Re: Any Way to SSH to a remote Hadoop Machine
Posted by Nitesh Bhatia <ni...@gmail.com>.
Just for experimental purpose you can setup 2 virtual machine on same host..
All you need to do it to install linux (Fed/Ubu) on 1 virtual machine using
Vmware/Xen/Virtualbox and configure hadoop either using Cloudera's Hadoop or
Apache Hadoop. Once its configured make your Current system on which you are
running this machine as Master and make this virtual machine its slave. Now
all you need to do is to copy-paste same virtual machine and add it to
vmware. Depending on your RAM / CPU Capacity you can work on any number of
machines. I believe it will solve your purpose.
If you are in college, I think its a positive thing for you. Since hadoop
does not require any high configuration servers, you can setup hadoop in
labs. All you need is a decent configuration PC with a linux :)
--nitesh
On Mon, Nov 16, 2009 at 2:16 PM, Jeff Hammerbacher <ha...@cloudera.com>wrote:
> Hey Prabhu,
>
> You may find the virtual machine available at
> http://www.cloudera.com/hadoop-training-virtual-machine to be of some use.
>
> Regards,
> Jeff
>
> On Sun, Nov 15, 2009 at 10:57 PM, Tim Robertson
> <ti...@gmail.com>wrote:
>
> > Some universities I think run clusters for teaching purposes so you
> > might consider investigating them.
> >
> > I'm afraid though, that you might need to start work on your own
> > system again. Ask questions to the list - be as specific and give as
> > much info as you can (post stack traces, and describe the hardware) -
> > and we will try and help as much as we can. It should not be too hard
> > to get you running in a day or so. Start by getting it running on a
> > single machine, then a pseudo distribution and then if you have more
> > machines, set up a cluster. Don't think you need a big cluster to
> > 'practice on'. I expect we all run on our laptops and small data
> > before running on a cluster.
> >
> > Cheers,
> > Tim
> >
> >
> >
> >
> > On Mon, Nov 16, 2009 at 7:49 AM, Prabhu Hari Dhanapal
> > <dr...@gmail.com> wrote:
> > > Thanks Tim !! I m still a student !
> > >
> > > so .. i was wondering is there a cost - effective option :)
> > >
> > > On Mon, Nov 16, 2009 at 1:37 AM, Tim Robertson <
> > timrobertson100@gmail.com>wrote:
> > >
> > >> You could try the Amazon Elastic MapReduce
> > >> (http://aws.amazon.com/elasticmapreduce/)
> > >> This is Hadoop, and costs less than US$1 per hour for a small cluster
> > >> - you only pay for what you use.
> > >>
> > >> Alternatively you could use the Amazon EC2 instances and run your own
> > >> cluster on their hardware also for similar $. Cloudera provide good
> > >> Amazon images to do this.
> > >>
> > >> Cheers
> > >> Tim
> > >>
> > >>
> > >> On Mon, Nov 16, 2009 at 7:28 AM, Prabhu Hari Dhanapal
> > >> <dr...@gmail.com> wrote:
> > >> > Hi,
> > >> >
> > >> > I have been having lot of problems installing hadoop on my system
> > and I
> > >> > finally gave up.I was just wondering if I could SSH to a remote
> hadoop
> > >> > cluster that is open for developers to practice on ? Is there
> anything
> > >> > provided by cloudera ? Please let me know.
> > >> >
> > >> > Thanks in Advance.
> > >> > --
> > >> > Hari
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Hari
> > >
> >
>
--
Nitesh Bhatia
"Life is never perfect. It just depends where you draw the line."
http://www.linkedin.com/in/niteshbhatia
http://www.twitter.com/niteshbhatia
Re: Any Way to SSH to a remote Hadoop Machine
Posted by Jeff Hammerbacher <ha...@cloudera.com>.
Hey Prabhu,
You may find the virtual machine available at
http://www.cloudera.com/hadoop-training-virtual-machine to be of some use.
Regards,
Jeff
On Sun, Nov 15, 2009 at 10:57 PM, Tim Robertson
<ti...@gmail.com>wrote:
> Some universities I think run clusters for teaching purposes so you
> might consider investigating them.
>
> I'm afraid though, that you might need to start work on your own
> system again. Ask questions to the list - be as specific and give as
> much info as you can (post stack traces, and describe the hardware) -
> and we will try and help as much as we can. It should not be too hard
> to get you running in a day or so. Start by getting it running on a
> single machine, then a pseudo distribution and then if you have more
> machines, set up a cluster. Don't think you need a big cluster to
> 'practice on'. I expect we all run on our laptops and small data
> before running on a cluster.
>
> Cheers,
> Tim
>
>
>
>
> On Mon, Nov 16, 2009 at 7:49 AM, Prabhu Hari Dhanapal
> <dr...@gmail.com> wrote:
> > Thanks Tim !! I m still a student !
> >
> > so .. i was wondering is there a cost - effective option :)
> >
> > On Mon, Nov 16, 2009 at 1:37 AM, Tim Robertson <
> timrobertson100@gmail.com>wrote:
> >
> >> You could try the Amazon Elastic MapReduce
> >> (http://aws.amazon.com/elasticmapreduce/)
> >> This is Hadoop, and costs less than US$1 per hour for a small cluster
> >> - you only pay for what you use.
> >>
> >> Alternatively you could use the Amazon EC2 instances and run your own
> >> cluster on their hardware also for similar $. Cloudera provide good
> >> Amazon images to do this.
> >>
> >> Cheers
> >> Tim
> >>
> >>
> >> On Mon, Nov 16, 2009 at 7:28 AM, Prabhu Hari Dhanapal
> >> <dr...@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > I have been having lot of problems installing hadoop on my system
> and I
> >> > finally gave up.I was just wondering if I could SSH to a remote hadoop
> >> > cluster that is open for developers to practice on ? Is there anything
> >> > provided by cloudera ? Please let me know.
> >> >
> >> > Thanks in Advance.
> >> > --
> >> > Hari
> >> >
> >>
> >
> >
> >
> > --
> > Hari
> >
>
Re: Any Way to SSH to a remote Hadoop Machine
Posted by Tim Robertson <ti...@gmail.com>.
Some universities I think run clusters for teaching purposes so you
might consider investigating them.
I'm afraid though, that you might need to start work on your own
system again. Ask questions to the list - be as specific and give as
much info as you can (post stack traces, and describe the hardware) -
and we will try and help as much as we can. It should not be too hard
to get you running in a day or so. Start by getting it running on a
single machine, then a pseudo distribution and then if you have more
machines, set up a cluster. Don't think you need a big cluster to
'practice on'. I expect we all run on our laptops and small data
before running on a cluster.
Cheers,
Tim
On Mon, Nov 16, 2009 at 7:49 AM, Prabhu Hari Dhanapal
<dr...@gmail.com> wrote:
> Thanks Tim !! I m still a student !
>
> so .. i was wondering is there a cost - effective option :)
>
> On Mon, Nov 16, 2009 at 1:37 AM, Tim Robertson <ti...@gmail.com>wrote:
>
>> You could try the Amazon Elastic MapReduce
>> (http://aws.amazon.com/elasticmapreduce/)
>> This is Hadoop, and costs less than US$1 per hour for a small cluster
>> - you only pay for what you use.
>>
>> Alternatively you could use the Amazon EC2 instances and run your own
>> cluster on their hardware also for similar $. Cloudera provide good
>> Amazon images to do this.
>>
>> Cheers
>> Tim
>>
>>
>> On Mon, Nov 16, 2009 at 7:28 AM, Prabhu Hari Dhanapal
>> <dr...@gmail.com> wrote:
>> > Hi,
>> >
>> > I have been having lot of problems installing hadoop on my system and I
>> > finally gave up.I was just wondering if I could SSH to a remote hadoop
>> > cluster that is open for developers to practice on ? Is there anything
>> > provided by cloudera ? Please let me know.
>> >
>> > Thanks in Advance.
>> > --
>> > Hari
>> >
>>
>
>
>
> --
> Hari
>
Re: Any Way to SSH to a remote Hadoop Machine
Posted by Prabhu Hari Dhanapal <dr...@gmail.com>.
Thanks Tim !! I m still a student !
so .. i was wondering is there a cost - effective option :)
On Mon, Nov 16, 2009 at 1:37 AM, Tim Robertson <ti...@gmail.com>wrote:
> You could try the Amazon Elastic MapReduce
> (http://aws.amazon.com/elasticmapreduce/)
> This is Hadoop, and costs less than US$1 per hour for a small cluster
> - you only pay for what you use.
>
> Alternatively you could use the Amazon EC2 instances and run your own
> cluster on their hardware also for similar $. Cloudera provide good
> Amazon images to do this.
>
> Cheers
> Tim
>
>
> On Mon, Nov 16, 2009 at 7:28 AM, Prabhu Hari Dhanapal
> <dr...@gmail.com> wrote:
> > Hi,
> >
> > I have been having lot of problems installing hadoop on my system and I
> > finally gave up.I was just wondering if I could SSH to a remote hadoop
> > cluster that is open for developers to practice on ? Is there anything
> > provided by cloudera ? Please let me know.
> >
> > Thanks in Advance.
> > --
> > Hari
> >
>
--
Hari
Re: Any Way to SSH to a remote Hadoop Machine
Posted by Tim Robertson <ti...@gmail.com>.
You could try the Amazon Elastic MapReduce
(http://aws.amazon.com/elasticmapreduce/)
This is Hadoop, and costs less than US$1 per hour for a small cluster
- you only pay for what you use.
Alternatively you could use the Amazon EC2 instances and run your own
cluster on their hardware also for similar $. Cloudera provide good
Amazon images to do this.
Cheers
Tim
On Mon, Nov 16, 2009 at 7:28 AM, Prabhu Hari Dhanapal
<dr...@gmail.com> wrote:
> Hi,
>
> I have been having lot of problems installing hadoop on my system and I
> finally gave up.I was just wondering if I could SSH to a remote hadoop
> cluster that is open for developers to practice on ? Is there anything
> provided by cloudera ? Please let me know.
>
> Thanks in Advance.
> --
> Hari
>
Re: Any Way to SSH to a remote Hadoop Machine
Posted by Prabhu Hari Dhanapal <dr...@gmail.com>.
Thanks Jeff !! But , is there a public or open cluster I could register
and get acess to ?
or any training cluster thats already setup which i could use?
On Mon, Nov 16, 2009 at 1:34 AM, Jeff Zhang <zj...@gmail.com> wrote:
> Of course, you can ssh to remote hadoop cluster.
>
> And also you can edit and compile your code outside cluster, and then
> submit
> your jar to cluster to execute your map reduce jobs.
>
>
> Jeff Zhang
>
>
> On Mon, Nov 16, 2009 at 2:28 PM, Prabhu Hari Dhanapal <
> dragonzsnake@gmail.com> wrote:
>
> > Hi,
> >
> > I have been having lot of problems installing hadoop on my system and I
> > finally gave up.I was just wondering if I could SSH to a remote hadoop
> > cluster that is open for developers to practice on ? Is there anything
> > provided by cloudera ? Please let me know.
> >
> > Thanks in Advance.
> > --
> > Hari
> >
>
--
Hari
Re: Any Way to SSH to a remote Hadoop Machine
Posted by Jeff Zhang <zj...@gmail.com>.
Of course, you can ssh to remote hadoop cluster.
And also you can edit and compile your code outside cluster, and then submit
your jar to cluster to execute your map reduce jobs.
Jeff Zhang
On Mon, Nov 16, 2009 at 2:28 PM, Prabhu Hari Dhanapal <
dragonzsnake@gmail.com> wrote:
> Hi,
>
> I have been having lot of problems installing hadoop on my system and I
> finally gave up.I was just wondering if I could SSH to a remote hadoop
> cluster that is open for developers to practice on ? Is there anything
> provided by cloudera ? Please let me know.
>
> Thanks in Advance.
> --
> Hari
>