You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ashish pareek <pa...@gmail.com> on 2009/06/18 16:46:51 UTC
Fwd: Need help
Hello,
I am doing my master.... my final year project is on Hadoop ...so I
would like to know some thing about Hadoop cluster i.e, Do new version of
Hadoop are able to handle heterogeneous hardware.If you have any
informantion regarding these please mail me as my project is in heterogenous
environment.
Thanks!
Reagrds,
Ashish Pareek
Re: Need help
Posted by ashish pareek <pa...@gmail.com>.
Hello Everybody,
How can we handle different applications having
different requirement being run on the same hadoop cluster ? What are the
various approaches to solve such problem.. if possible please mention some
of those ideas.
Does such implementation exists ?
Thanks ,
Regards,
Ashish
On Thu, Jun 18, 2009 at 9:36 PM, jason hadoop <ja...@gmail.com>wrote:
> For me, I like to have one configuration file that I distribute to all of
> the machines in my cluster via rsync.
>
> In there are things like the number of tasks per node to run, and where to
> store dfs data and local temporary data, and the limits to storage for the
> machines.
>
> If the machines are very different, it becomes important to tailor the
> configuration file per machine or type of machine.
>
> At this point, you are pretty much going to have to spend the time, reading
> through the details of configuring a hadoop cluster.
>
>
> On Thu, Jun 18, 2009 at 8:33 AM, ashish pareek <pa...@gmail.com>
> wrote:
>
> > Can you tell few of the challenges in configuring heterogeneous
> > cluster...or
> > can pass on some link where I would get some information regarding
> > challenges in running Hadoop on heterogeneous hardware
> >
> > One more things is How about running different applications on the same
> > Hadoop cluster?and what challenges are involved in it ?
> >
> > Thanks,
> > Regards,
> > Ashish
> >
> >
> > On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop <jason.hadoop@gmail.com
> > >wrote:
> >
> > > I don't know anyone who has a completely homogeneous cluster.
> > >
> > > So hadoop is scalable across heterogeneous environments.
> > >
> > > I stated that configuration is simpler if the machines are similar
> (There
> > > are optimizations in configuration for near homogeneous machines.)
> > >
> > > On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pa...@gmail.com>
> > > wrote:
> > >
> > > > Does that mean hadoop is not scalable wrt heterogeneous environment?
> > and
> > > > one
> > > > more question is can we run different application on the same hadoop
> > > > cluster
> > > > .
> > > >
> > > > Thanks.
> > > > Regards,
> > > > Ashish
> > > >
> > > > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <
> jason.hadoop@gmail.com
> > > > >wrote:
> > > >
> > > > > Hadoop has always been reasonably agnostic wrt hardware and
> > > homogeneity.
> > > > > There are optimizations in configuration for near homogeneous
> > machines.
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <
> pareekash@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > > I am doing my master.... my final year project is on
> Hadoop
> > > > ...so
> > > > > I
> > > > > > would like to know some thing about Hadoop cluster i.e, Do new
> > > version
> > > > of
> > > > > > Hadoop are able to handle heterogeneous hardware.If you have any
> > > > > > informantion regarding these please mail me as my project is in
> > > > > > heterogenous
> > > > > > environment.
> > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Reagrds,
> > > > > > Ashish Pareek
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > > > www.prohadoopbook.com a community for Hadoop Professionals
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > www.prohadoopbook.com a community for Hadoop Professionals
> > >
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
Re: Need help
Posted by ashish pareek <pa...@gmail.com>.
Hello Everybody,
How can we handle different applications having
different requirement being run on the same hadoop cluster ? What are the
various approaches to solve such problem.. if possible please mention some
of those ideas.
Does such implementation exists ?
Thanks ,
Regards,
Ashish
On Thu, Jun 18, 2009 at 9:36 PM, jason hadoop <ja...@gmail.com>wrote:
> For me, I like to have one configuration file that I distribute to all of
> the machines in my cluster via rsync.
>
> In there are things like the number of tasks per node to run, and where to
> store dfs data and local temporary data, and the limits to storage for the
> machines.
>
> If the machines are very different, it becomes important to tailor the
> configuration file per machine or type of machine.
>
> At this point, you are pretty much going to have to spend the time, reading
> through the details of configuring a hadoop cluster.
>
>
> On Thu, Jun 18, 2009 at 8:33 AM, ashish pareek <pa...@gmail.com>
> wrote:
>
> > Can you tell few of the challenges in configuring heterogeneous
> > cluster...or
> > can pass on some link where I would get some information regarding
> > challenges in running Hadoop on heterogeneous hardware
> >
> > One more things is How about running different applications on the same
> > Hadoop cluster?and what challenges are involved in it ?
> >
> > Thanks,
> > Regards,
> > Ashish
> >
> >
> > On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop <jason.hadoop@gmail.com
> > >wrote:
> >
> > > I don't know anyone who has a completely homogeneous cluster.
> > >
> > > So hadoop is scalable across heterogeneous environments.
> > >
> > > I stated that configuration is simpler if the machines are similar
> (There
> > > are optimizations in configuration for near homogeneous machines.)
> > >
> > > On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pa...@gmail.com>
> > > wrote:
> > >
> > > > Does that mean hadoop is not scalable wrt heterogeneous environment?
> > and
> > > > one
> > > > more question is can we run different application on the same hadoop
> > > > cluster
> > > > .
> > > >
> > > > Thanks.
> > > > Regards,
> > > > Ashish
> > > >
> > > > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <
> jason.hadoop@gmail.com
> > > > >wrote:
> > > >
> > > > > Hadoop has always been reasonably agnostic wrt hardware and
> > > homogeneity.
> > > > > There are optimizations in configuration for near homogeneous
> > machines.
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <
> pareekash@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > > I am doing my master.... my final year project is on
> Hadoop
> > > > ...so
> > > > > I
> > > > > > would like to know some thing about Hadoop cluster i.e, Do new
> > > version
> > > > of
> > > > > > Hadoop are able to handle heterogeneous hardware.If you have any
> > > > > > informantion regarding these please mail me as my project is in
> > > > > > heterogenous
> > > > > > environment.
> > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Reagrds,
> > > > > > Ashish Pareek
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > > > www.prohadoopbook.com a community for Hadoop Professionals
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > www.prohadoopbook.com a community for Hadoop Professionals
> > >
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
Re: Need help
Posted by jason hadoop <ja...@gmail.com>.
For me, I like to have one configuration file that I distribute to all of
the machines in my cluster via rsync.
In there are things like the number of tasks per node to run, and where to
store dfs data and local temporary data, and the limits to storage for the
machines.
If the machines are very different, it becomes important to tailor the
configuration file per machine or type of machine.
At this point, you are pretty much going to have to spend the time, reading
through the details of configuring a hadoop cluster.
On Thu, Jun 18, 2009 at 8:33 AM, ashish pareek <pa...@gmail.com> wrote:
> Can you tell few of the challenges in configuring heterogeneous
> cluster...or
> can pass on some link where I would get some information regarding
> challenges in running Hadoop on heterogeneous hardware
>
> One more things is How about running different applications on the same
> Hadoop cluster?and what challenges are involved in it ?
>
> Thanks,
> Regards,
> Ashish
>
>
> On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop <jason.hadoop@gmail.com
> >wrote:
>
> > I don't know anyone who has a completely homogeneous cluster.
> >
> > So hadoop is scalable across heterogeneous environments.
> >
> > I stated that configuration is simpler if the machines are similar (There
> > are optimizations in configuration for near homogeneous machines.)
> >
> > On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pa...@gmail.com>
> > wrote:
> >
> > > Does that mean hadoop is not scalable wrt heterogeneous environment?
> and
> > > one
> > > more question is can we run different application on the same hadoop
> > > cluster
> > > .
> > >
> > > Thanks.
> > > Regards,
> > > Ashish
> > >
> > > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <jason.hadoop@gmail.com
> > > >wrote:
> > >
> > > > Hadoop has always been reasonably agnostic wrt hardware and
> > homogeneity.
> > > > There are optimizations in configuration for near homogeneous
> machines.
> > > >
> > > >
> > > >
> > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <pa...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello,
> > > > > I am doing my master.... my final year project is on Hadoop
> > > ...so
> > > > I
> > > > > would like to know some thing about Hadoop cluster i.e, Do new
> > version
> > > of
> > > > > Hadoop are able to handle heterogeneous hardware.If you have any
> > > > > informantion regarding these please mail me as my project is in
> > > > > heterogenous
> > > > > environment.
> > > > >
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Reagrds,
> > > > > Ashish Pareek
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > > www.prohadoopbook.com a community for Hadoop Professionals
> > > >
> > >
> >
> >
> >
> > --
> > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > www.prohadoopbook.com a community for Hadoop Professionals
> >
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals
Re: Need help
Posted by Matt Massie <ma...@cloudera.com>.
Hadoop can be run on a hardware heterogeneous cluster. Currently,
Hadoop clusters really only run well on Linux although you can run a
Hadoop client on non-Linux machines.
You will need to have a special configuration for each of the machine
in your cluster based on their hardware profile. Ideally, you'll be
able to group the machines in your cluster into "classes" of machines
(e.g. machines with 1GB of RAM and 2 core versus 4GB of RAM and 4
core) to reduce the burden of managing multiple configurations. If
you are talking about a Hadoop cluster that is completely
heterogeneous (each machine is completely different), the management
overhead could be high.
Configuration variables like "mapred.tasktracker.map.tasks.maximum"
and "mapred.tasktracker.reduce.tasks.maximum" should be set based on
the number of cores/memory in each machine. Variables like
"mapred.child.java.opts" need to be set differently based on the
amount of memory the machine has (e.g. "-Xmx250m"). You should have
at least 250MB of memory dedicated to each task although more is
better. It's also wise to make sure that each task has the same
amount of memory regardless of the machine it's scheduled on;
otherwise, tasks might succeed or fail based on which machine gets the
task. This asymmetry will make debugging harder.
You can use our online configurator (http://www.cloudera.com/configurator/
), to generate optimized configurations for each class of machines in
your cluster. It will ask simple question about your configuration
and then produce a hadoop-site.xml file.
Good luck!
-Matt
On Jun 18, 2009, at 8:33 AM, ashish pareek wrote:
> Can you tell few of the challenges in configuring heterogeneous
> cluster...or
> can pass on some link where I would get some information regarding
> challenges in running Hadoop on heterogeneous hardware
>
> One more things is How about running different applications on the
> same
> Hadoop cluster?and what challenges are involved in it ?
>
> Thanks,
> Regards,
> Ashish
>
>
> On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop
> <ja...@gmail.com>wrote:
>
>> I don't know anyone who has a completely homogeneous cluster.
>>
>> So hadoop is scalable across heterogeneous environments.
>>
>> I stated that configuration is simpler if the machines are similar
>> (There
>> are optimizations in configuration for near homogeneous machines.)
>>
>> On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pa...@gmail.com>
>> wrote:
>>
>>> Does that mean hadoop is not scalable wrt heterogeneous
>>> environment? and
>>> one
>>> more question is can we run different application on the same hadoop
>>> cluster
>>> .
>>>
>>> Thanks.
>>> Regards,
>>> Ashish
>>>
>>> On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop
>>> <jason.hadoop@gmail.com
>>>> wrote:
>>>
>>>> Hadoop has always been reasonably agnostic wrt hardware and
>> homogeneity.
>>>> There are optimizations in configuration for near homogeneous
>>>> machines.
>>>>
>>>>
>>>>
>>>> On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek
>>>> <pa...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>> I am doing my master.... my final year project is on Hadoop
>>> ...so
>>>> I
>>>>> would like to know some thing about Hadoop cluster i.e, Do new
>> version
>>> of
>>>>> Hadoop are able to handle heterogeneous hardware.If you have any
>>>>> informantion regarding these please mail me as my project is in
>>>>> heterogenous
>>>>> environment.
>>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Reagrds,
>>>>> Ashish Pareek
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
>>>> http://www.amazon.com/dp/1430219424?tag=jewlerymall
>>>> www.prohadoopbook.com a community for Hadoop Professionals
>>>>
>>>
>>
>>
>>
>> --
>> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
>> http://www.amazon.com/dp/1430219424?tag=jewlerymall
>> www.prohadoopbook.com a community for Hadoop Professionals
>>
Re: Need help
Posted by Matt Massie <ma...@cloudera.com>.
Hadoop can be run on a hardware heterogeneous cluster. Currently,
Hadoop clusters really only run well on Linux although you can run a
Hadoop client on non-Linux machines.
You will need to have a special configuration for each of the machine
in your cluster based on their hardware profile. Ideally, you'll be
able to group the machines in your cluster into "classes" of machines
(e.g. machines with 1GB of RAM and 2 core versus 4GB of RAM and 4
core) to reduce the burden of managing multiple configurations. If
you are talking about a Hadoop cluster that is completely
heterogeneous (each machine is completely different), the management
overhead could be high.
Configuration variables like "mapred.tasktracker.map.tasks.maximum"
and "mapred.tasktracker.reduce.tasks.maximum" should be set based on
the number of cores/memory in each machine. Variables like
"mapred.child.java.opts" need to be set differently based on the
amount of memory the machine has (e.g. "-Xmx250m"). You should have
at least 250MB of memory dedicated to each task although more is
better. It's also wise to make sure that each task has the same
amount of memory regardless of the machine it's scheduled on;
otherwise, tasks might succeed or fail based on which machine gets the
task. This asymmetry will make debugging harder.
You can use our online configurator (http://www.cloudera.com/configurator/
), to generate optimized configurations for each class of machines in
your cluster. It will ask simple question about your configuration
and then produce a hadoop-site.xml file.
Good luck!
-Matt
On Jun 18, 2009, at 8:33 AM, ashish pareek wrote:
> Can you tell few of the challenges in configuring heterogeneous
> cluster...or
> can pass on some link where I would get some information regarding
> challenges in running Hadoop on heterogeneous hardware
>
> One more things is How about running different applications on the
> same
> Hadoop cluster?and what challenges are involved in it ?
>
> Thanks,
> Regards,
> Ashish
>
>
> On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop
> <ja...@gmail.com>wrote:
>
>> I don't know anyone who has a completely homogeneous cluster.
>>
>> So hadoop is scalable across heterogeneous environments.
>>
>> I stated that configuration is simpler if the machines are similar
>> (There
>> are optimizations in configuration for near homogeneous machines.)
>>
>> On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pa...@gmail.com>
>> wrote:
>>
>>> Does that mean hadoop is not scalable wrt heterogeneous
>>> environment? and
>>> one
>>> more question is can we run different application on the same hadoop
>>> cluster
>>> .
>>>
>>> Thanks.
>>> Regards,
>>> Ashish
>>>
>>> On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop
>>> <jason.hadoop@gmail.com
>>>> wrote:
>>>
>>>> Hadoop has always been reasonably agnostic wrt hardware and
>> homogeneity.
>>>> There are optimizations in configuration for near homogeneous
>>>> machines.
>>>>
>>>>
>>>>
>>>> On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek
>>>> <pa...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>> I am doing my master.... my final year project is on Hadoop
>>> ...so
>>>> I
>>>>> would like to know some thing about Hadoop cluster i.e, Do new
>> version
>>> of
>>>>> Hadoop are able to handle heterogeneous hardware.If you have any
>>>>> informantion regarding these please mail me as my project is in
>>>>> heterogenous
>>>>> environment.
>>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Reagrds,
>>>>> Ashish Pareek
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
>>>> http://www.amazon.com/dp/1430219424?tag=jewlerymall
>>>> www.prohadoopbook.com a community for Hadoop Professionals
>>>>
>>>
>>
>>
>>
>> --
>> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
>> http://www.amazon.com/dp/1430219424?tag=jewlerymall
>> www.prohadoopbook.com a community for Hadoop Professionals
>>
Re: Need help
Posted by ashish pareek <pa...@gmail.com>.
Can you tell few of the challenges in configuring heterogeneous cluster...or
can pass on some link where I would get some information regarding
challenges in running Hadoop on heterogeneous hardware
One more things is How about running different applications on the same
Hadoop cluster?and what challenges are involved in it ?
Thanks,
Regards,
Ashish
On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop <ja...@gmail.com>wrote:
> I don't know anyone who has a completely homogeneous cluster.
>
> So hadoop is scalable across heterogeneous environments.
>
> I stated that configuration is simpler if the machines are similar (There
> are optimizations in configuration for near homogeneous machines.)
>
> On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pa...@gmail.com>
> wrote:
>
> > Does that mean hadoop is not scalable wrt heterogeneous environment? and
> > one
> > more question is can we run different application on the same hadoop
> > cluster
> > .
> >
> > Thanks.
> > Regards,
> > Ashish
> >
> > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <jason.hadoop@gmail.com
> > >wrote:
> >
> > > Hadoop has always been reasonably agnostic wrt hardware and
> homogeneity.
> > > There are optimizations in configuration for near homogeneous machines.
> > >
> > >
> > >
> > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <pa...@gmail.com>
> > > wrote:
> > >
> > > > Hello,
> > > > I am doing my master.... my final year project is on Hadoop
> > ...so
> > > I
> > > > would like to know some thing about Hadoop cluster i.e, Do new
> version
> > of
> > > > Hadoop are able to handle heterogeneous hardware.If you have any
> > > > informantion regarding these please mail me as my project is in
> > > > heterogenous
> > > > environment.
> > > >
> > > >
> > > > Thanks!
> > > >
> > > > Reagrds,
> > > > Ashish Pareek
> > > >
> > >
> > >
> > >
> > > --
> > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > www.prohadoopbook.com a community for Hadoop Professionals
> > >
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
Re: Need help
Posted by ashish pareek <pa...@gmail.com>.
Can you tell few of the challenges in configuring heterogeneous cluster...or
can pass on some link where I would get some information regarding
challenges in running Hadoop on heterogeneous hardware
One more things is How about running different applications on the same
Hadoop cluster?and what challenges are involved in it ?
Thanks,
Regards,
Ashish
On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop <ja...@gmail.com>wrote:
> I don't know anyone who has a completely homogeneous cluster.
>
> So hadoop is scalable across heterogeneous environments.
>
> I stated that configuration is simpler if the machines are similar (There
> are optimizations in configuration for near homogeneous machines.)
>
> On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pa...@gmail.com>
> wrote:
>
> > Does that mean hadoop is not scalable wrt heterogeneous environment? and
> > one
> > more question is can we run different application on the same hadoop
> > cluster
> > .
> >
> > Thanks.
> > Regards,
> > Ashish
> >
> > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <jason.hadoop@gmail.com
> > >wrote:
> >
> > > Hadoop has always been reasonably agnostic wrt hardware and
> homogeneity.
> > > There are optimizations in configuration for near homogeneous machines.
> > >
> > >
> > >
> > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <pa...@gmail.com>
> > > wrote:
> > >
> > > > Hello,
> > > > I am doing my master.... my final year project is on Hadoop
> > ...so
> > > I
> > > > would like to know some thing about Hadoop cluster i.e, Do new
> version
> > of
> > > > Hadoop are able to handle heterogeneous hardware.If you have any
> > > > informantion regarding these please mail me as my project is in
> > > > heterogenous
> > > > environment.
> > > >
> > > >
> > > > Thanks!
> > > >
> > > > Reagrds,
> > > > Ashish Pareek
> > > >
> > >
> > >
> > >
> > > --
> > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > www.prohadoopbook.com a community for Hadoop Professionals
> > >
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
Re: Need help
Posted by jason hadoop <ja...@gmail.com>.
I don't know anyone who has a completely homogeneous cluster.
So hadoop is scalable across heterogeneous environments.
I stated that configuration is simpler if the machines are similar (There
are optimizations in configuration for near homogeneous machines.)
On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pa...@gmail.com> wrote:
> Does that mean hadoop is not scalable wrt heterogeneous environment? and
> one
> more question is can we run different application on the same hadoop
> cluster
> .
>
> Thanks.
> Regards,
> Ashish
>
> On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <jason.hadoop@gmail.com
> >wrote:
>
> > Hadoop has always been reasonably agnostic wrt hardware and homogeneity.
> > There are optimizations in configuration for near homogeneous machines.
> >
> >
> >
> > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <pa...@gmail.com>
> > wrote:
> >
> > > Hello,
> > > I am doing my master.... my final year project is on Hadoop
> ...so
> > I
> > > would like to know some thing about Hadoop cluster i.e, Do new version
> of
> > > Hadoop are able to handle heterogeneous hardware.If you have any
> > > informantion regarding these please mail me as my project is in
> > > heterogenous
> > > environment.
> > >
> > >
> > > Thanks!
> > >
> > > Reagrds,
> > > Ashish Pareek
> > >
> >
> >
> >
> > --
> > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > www.prohadoopbook.com a community for Hadoop Professionals
> >
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals
Re: Need help
Posted by ashish pareek <pa...@gmail.com>.
Does that mean hadoop is not scalable wrt heterogeneous environment? and one
more question is can we run different application on the same hadoop cluster
.
Thanks.
Regards,
Ashish
On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <ja...@gmail.com>wrote:
> Hadoop has always been reasonably agnostic wrt hardware and homogeneity.
> There are optimizations in configuration for near homogeneous machines.
>
>
>
> On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <pa...@gmail.com>
> wrote:
>
> > Hello,
> > I am doing my master.... my final year project is on Hadoop ...so
> I
> > would like to know some thing about Hadoop cluster i.e, Do new version of
> > Hadoop are able to handle heterogeneous hardware.If you have any
> > informantion regarding these please mail me as my project is in
> > heterogenous
> > environment.
> >
> >
> > Thanks!
> >
> > Reagrds,
> > Ashish Pareek
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
Re: Need help
Posted by ashish pareek <pa...@gmail.com>.
Does that mean hadoop is not scalable wrt heterogeneous environment? and one
more question is can we run different application on the same hadoop cluster
.
Thanks.
Regards,
Ashish
On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <ja...@gmail.com>wrote:
> Hadoop has always been reasonably agnostic wrt hardware and homogeneity.
> There are optimizations in configuration for near homogeneous machines.
>
>
>
> On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <pa...@gmail.com>
> wrote:
>
> > Hello,
> > I am doing my master.... my final year project is on Hadoop ...so
> I
> > would like to know some thing about Hadoop cluster i.e, Do new version of
> > Hadoop are able to handle heterogeneous hardware.If you have any
> > informantion regarding these please mail me as my project is in
> > heterogenous
> > environment.
> >
> >
> > Thanks!
> >
> > Reagrds,
> > Ashish Pareek
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
Re: Need help
Posted by jason hadoop <ja...@gmail.com>.
Hadoop has always been reasonably agnostic wrt hardware and homogeneity.
There are optimizations in configuration for near homogeneous machines.
On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <pa...@gmail.com> wrote:
> Hello,
> I am doing my master.... my final year project is on Hadoop ...so I
> would like to know some thing about Hadoop cluster i.e, Do new version of
> Hadoop are able to handle heterogeneous hardware.If you have any
> informantion regarding these please mail me as my project is in
> heterogenous
> environment.
>
>
> Thanks!
>
> Reagrds,
> Ashish Pareek
>
--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals