You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by jing wang <ha...@gmail.com> on 2013/02/19 09:03:03 UTC
CDH4 version questions
Hi User,
We're facing the challenge of which hadoop version to choose.
We prefer to CDH4, but have a few qustions:
1.Are MRV1 and MRV2 sharing the same hdfs? If so, can MRV1 upgrade to
MRV2 smoothly?
2.If using MRV1, should our m/r code basing CDH3 be changed?
3.Is MRV1 stable enough to be used in production?
Thanks & best regards,
Jing Wang
Re: CDH4 version questions
Posted by Arun C Murthy <ac...@hortonworks.com>.
Pls ask CDH lists.
On Feb 19, 2013, at 12:03 AM, jing wang wrote:
> Hi User,
>
> We're facing the challenge of which hadoop version to choose.
> We prefer to CDH4, but have a few qustions:
> 1.Are MRV1 and MRV2 sharing the same hdfs? If so, can MRV1 upgrade to MRV2 smoothly?
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
> 3.Is MRV1 stable enough to be used in production?
>
>
> Thanks & best regards,
> Jing Wang
>
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: CDH4 version questions
Posted by jing wang <ha...@gmail.com>.
Hi David,
Thanks for your reply.
Some more questions about installing MRV1 using tarball.
For the link '
https://ccp.cloudera.com/display/SUPPORT/CDH4+Downloadable+Tarballs',
should I intall MRV1 using 'hadoop-0.20-mapreduce-0.20.2+1270' or
'hadoop-2.0.0+556' or both of them (
If you install CDH4 from a tarball, you will install YARN. To install MRv1
as well, install the separate MRv1 tarball (mr1-0.20.2+<n>) alongside the
YARN one (hadoop-2.0.0+<n>).
in '
https://ccp.cloudera.com/display/CDH4DOC/CDH4+Installation#CDH4Installation-WaysToInstallCDH4'
)?
If using 'hadoop-0.20-mapreduce-0.20.2+1270', are the configurations same
like CDH3?
If not, would you please show me some link or docs to install MRV1?
Thanks & Best Regards
Jing Wang
2013/2/19 David Boyd <db...@lorenzresearch.com>
> Jing:
> We are using CDH4.1.1 and CDH3 and the
> base Apache Hadoop on several different efforts.
>
> My best cut at answering your questions is below
> however the CDH lists will likely have better
> information on some of them.
>
>
> On 2/19/2013 3:03 AM, jing wang wrote:
>
>> Hi User,
>>
>> We're facing the challenge of which hadoop version to choose.
>> We prefer to CDH4, but have a few qustions:
>> 1.Are MRV1 and MRV2 sharing the same hdfs?
>>
> YES.
>
>
> If so, can MRV1 upgrade
>
>> to MRV2 smoothly?
>>
> They should but we have not played with MRV2 much yet.
>
>
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
>>
> Some. You will certainly get deprecated warnings. Biggest issue
> seems to be getting the right CDH4 libraries in the path.
>
>
> 3.Is MRV1 stable enough to be used in production?
>>
> Yes. I have been beating on a CDH4 with MRV1 code heavily.
> This includes running the SWIM benchmark with the Facebook
> workload.
>
> Even cloudera does not thing MRV2 is quite stable enough yet.
>
>
>
>>
>> Thanks & best regards,
>> Jing Wang
>>
>>
>>
>>
> --
> ========= mailto:dboyd@lorenzresearch.**com <db...@lorenzresearch.com>============
> David W. Boyd
> Vice President, Operations
> Lorenz Research, a Data Tactics corporation
> 7901 Jones Branch, Suite 610
> Mclean, VA 22102
> office: +1-703-506-3735, ext 308
> fax: +1-703-506-6703
> cell: +1-703-402-7908
> ============== http://www.lorenzresearch.com/ ============
>
>
> The information contained in this message may be privileged
> and/or confidential and protected from disclosure.
> If the reader of this message is not the intended recipient
> or an employee or agent responsible for delivering this message
> to the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication
> is strictly prohibited. If you have received this communication
> in error, please notify the sender immediately by replying to
> this message and deleting the material from any computer.
>
>
>
Re: CDH4 version questions
Posted by jing wang <ha...@gmail.com>.
Hi David,
Thanks for your reply.
Some more questions about installing MRV1 using tarball.
For the link '
https://ccp.cloudera.com/display/SUPPORT/CDH4+Downloadable+Tarballs',
should I intall MRV1 using 'hadoop-0.20-mapreduce-0.20.2+1270' or
'hadoop-2.0.0+556' or both of them (
If you install CDH4 from a tarball, you will install YARN. To install MRv1
as well, install the separate MRv1 tarball (mr1-0.20.2+<n>) alongside the
YARN one (hadoop-2.0.0+<n>).
in '
https://ccp.cloudera.com/display/CDH4DOC/CDH4+Installation#CDH4Installation-WaysToInstallCDH4'
)?
If using 'hadoop-0.20-mapreduce-0.20.2+1270', are the configurations same
like CDH3?
If not, would you please show me some link or docs to install MRV1?
Thanks & Best Regards
Jing Wang
2013/2/19 David Boyd <db...@lorenzresearch.com>
> Jing:
> We are using CDH4.1.1 and CDH3 and the
> base Apache Hadoop on several different efforts.
>
> My best cut at answering your questions is below
> however the CDH lists will likely have better
> information on some of them.
>
>
> On 2/19/2013 3:03 AM, jing wang wrote:
>
>> Hi User,
>>
>> We're facing the challenge of which hadoop version to choose.
>> We prefer to CDH4, but have a few qustions:
>> 1.Are MRV1 and MRV2 sharing the same hdfs?
>>
> YES.
>
>
> If so, can MRV1 upgrade
>
>> to MRV2 smoothly?
>>
> They should but we have not played with MRV2 much yet.
>
>
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
>>
> Some. You will certainly get deprecated warnings. Biggest issue
> seems to be getting the right CDH4 libraries in the path.
>
>
> 3.Is MRV1 stable enough to be used in production?
>>
> Yes. I have been beating on a CDH4 with MRV1 code heavily.
> This includes running the SWIM benchmark with the Facebook
> workload.
>
> Even cloudera does not thing MRV2 is quite stable enough yet.
>
>
>
>>
>> Thanks & best regards,
>> Jing Wang
>>
>>
>>
>>
> --
> ========= mailto:dboyd@lorenzresearch.**com <db...@lorenzresearch.com>============
> David W. Boyd
> Vice President, Operations
> Lorenz Research, a Data Tactics corporation
> 7901 Jones Branch, Suite 610
> Mclean, VA 22102
> office: +1-703-506-3735, ext 308
> fax: +1-703-506-6703
> cell: +1-703-402-7908
> ============== http://www.lorenzresearch.com/ ============
>
>
> The information contained in this message may be privileged
> and/or confidential and protected from disclosure.
> If the reader of this message is not the intended recipient
> or an employee or agent responsible for delivering this message
> to the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication
> is strictly prohibited. If you have received this communication
> in error, please notify the sender immediately by replying to
> this message and deleting the material from any computer.
>
>
>
Re: CDH4 version questions
Posted by jing wang <ha...@gmail.com>.
Hi David,
Thanks for your reply.
Some more questions about installing MRV1 using tarball.
For the link '
https://ccp.cloudera.com/display/SUPPORT/CDH4+Downloadable+Tarballs',
should I intall MRV1 using 'hadoop-0.20-mapreduce-0.20.2+1270' or
'hadoop-2.0.0+556' or both of them (
If you install CDH4 from a tarball, you will install YARN. To install MRv1
as well, install the separate MRv1 tarball (mr1-0.20.2+<n>) alongside the
YARN one (hadoop-2.0.0+<n>).
in '
https://ccp.cloudera.com/display/CDH4DOC/CDH4+Installation#CDH4Installation-WaysToInstallCDH4'
)?
If using 'hadoop-0.20-mapreduce-0.20.2+1270', are the configurations same
like CDH3?
If not, would you please show me some link or docs to install MRV1?
Thanks & Best Regards
Jing Wang
2013/2/19 David Boyd <db...@lorenzresearch.com>
> Jing:
> We are using CDH4.1.1 and CDH3 and the
> base Apache Hadoop on several different efforts.
>
> My best cut at answering your questions is below
> however the CDH lists will likely have better
> information on some of them.
>
>
> On 2/19/2013 3:03 AM, jing wang wrote:
>
>> Hi User,
>>
>> We're facing the challenge of which hadoop version to choose.
>> We prefer to CDH4, but have a few qustions:
>> 1.Are MRV1 and MRV2 sharing the same hdfs?
>>
> YES.
>
>
> If so, can MRV1 upgrade
>
>> to MRV2 smoothly?
>>
> They should but we have not played with MRV2 much yet.
>
>
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
>>
> Some. You will certainly get deprecated warnings. Biggest issue
> seems to be getting the right CDH4 libraries in the path.
>
>
> 3.Is MRV1 stable enough to be used in production?
>>
> Yes. I have been beating on a CDH4 with MRV1 code heavily.
> This includes running the SWIM benchmark with the Facebook
> workload.
>
> Even cloudera does not thing MRV2 is quite stable enough yet.
>
>
>
>>
>> Thanks & best regards,
>> Jing Wang
>>
>>
>>
>>
> --
> ========= mailto:dboyd@lorenzresearch.**com <db...@lorenzresearch.com>============
> David W. Boyd
> Vice President, Operations
> Lorenz Research, a Data Tactics corporation
> 7901 Jones Branch, Suite 610
> Mclean, VA 22102
> office: +1-703-506-3735, ext 308
> fax: +1-703-506-6703
> cell: +1-703-402-7908
> ============== http://www.lorenzresearch.com/ ============
>
>
> The information contained in this message may be privileged
> and/or confidential and protected from disclosure.
> If the reader of this message is not the intended recipient
> or an employee or agent responsible for delivering this message
> to the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication
> is strictly prohibited. If you have received this communication
> in error, please notify the sender immediately by replying to
> this message and deleting the material from any computer.
>
>
>
Re: CDH4 version questions
Posted by jing wang <ha...@gmail.com>.
Hi David,
Thanks for your reply.
Some more questions about installing MRV1 using tarball.
For the link '
https://ccp.cloudera.com/display/SUPPORT/CDH4+Downloadable+Tarballs',
should I intall MRV1 using 'hadoop-0.20-mapreduce-0.20.2+1270' or
'hadoop-2.0.0+556' or both of them (
If you install CDH4 from a tarball, you will install YARN. To install MRv1
as well, install the separate MRv1 tarball (mr1-0.20.2+<n>) alongside the
YARN one (hadoop-2.0.0+<n>).
in '
https://ccp.cloudera.com/display/CDH4DOC/CDH4+Installation#CDH4Installation-WaysToInstallCDH4'
)?
If using 'hadoop-0.20-mapreduce-0.20.2+1270', are the configurations same
like CDH3?
If not, would you please show me some link or docs to install MRV1?
Thanks & Best Regards
Jing Wang
2013/2/19 David Boyd <db...@lorenzresearch.com>
> Jing:
> We are using CDH4.1.1 and CDH3 and the
> base Apache Hadoop on several different efforts.
>
> My best cut at answering your questions is below
> however the CDH lists will likely have better
> information on some of them.
>
>
> On 2/19/2013 3:03 AM, jing wang wrote:
>
>> Hi User,
>>
>> We're facing the challenge of which hadoop version to choose.
>> We prefer to CDH4, but have a few qustions:
>> 1.Are MRV1 and MRV2 sharing the same hdfs?
>>
> YES.
>
>
> If so, can MRV1 upgrade
>
>> to MRV2 smoothly?
>>
> They should but we have not played with MRV2 much yet.
>
>
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
>>
> Some. You will certainly get deprecated warnings. Biggest issue
> seems to be getting the right CDH4 libraries in the path.
>
>
> 3.Is MRV1 stable enough to be used in production?
>>
> Yes. I have been beating on a CDH4 with MRV1 code heavily.
> This includes running the SWIM benchmark with the Facebook
> workload.
>
> Even cloudera does not thing MRV2 is quite stable enough yet.
>
>
>
>>
>> Thanks & best regards,
>> Jing Wang
>>
>>
>>
>>
> --
> ========= mailto:dboyd@lorenzresearch.**com <db...@lorenzresearch.com>============
> David W. Boyd
> Vice President, Operations
> Lorenz Research, a Data Tactics corporation
> 7901 Jones Branch, Suite 610
> Mclean, VA 22102
> office: +1-703-506-3735, ext 308
> fax: +1-703-506-6703
> cell: +1-703-402-7908
> ============== http://www.lorenzresearch.com/ ============
>
>
> The information contained in this message may be privileged
> and/or confidential and protected from disclosure.
> If the reader of this message is not the intended recipient
> or an employee or agent responsible for delivering this message
> to the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication
> is strictly prohibited. If you have received this communication
> in error, please notify the sender immediately by replying to
> this message and deleting the material from any computer.
>
>
>
Re: CDH4 version questions
Posted by David Boyd <db...@lorenzresearch.com>.
Jing:
We are using CDH4.1.1 and CDH3 and the
base Apache Hadoop on several different efforts.
My best cut at answering your questions is below
however the CDH lists will likely have better
information on some of them.
On 2/19/2013 3:03 AM, jing wang wrote:
> Hi User,
>
> We're facing the challenge of which hadoop version to choose.
> We prefer to CDH4, but have a few qustions:
> 1.Are MRV1 and MRV2 sharing the same hdfs?
YES.
If so, can MRV1 upgrade
> to MRV2 smoothly?
They should but we have not played with MRV2 much yet.
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
Some. You will certainly get deprecated warnings. Biggest issue
seems to be getting the right CDH4 libraries in the path.
> 3.Is MRV1 stable enough to be used in production?
Yes. I have been beating on a CDH4 with MRV1 code heavily.
This includes running the SWIM benchmark with the Facebook
workload.
Even cloudera does not thing MRV2 is quite stable enough yet.
>
>
> Thanks & best regards,
> Jing Wang
>
>
>
--
========= mailto:dboyd@lorenzresearch.com ============
David W. Boyd
Vice President, Operations
Lorenz Research, a Data Tactics corporation
7901 Jones Branch, Suite 610
Mclean, VA 22102
office: +1-703-506-3735, ext 308
fax: +1-703-506-6703
cell: +1-703-402-7908
============== http://www.lorenzresearch.com/ ============
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.
Re: CDH4 version questions
Posted by Arun C Murthy <ac...@hortonworks.com>.
Pls ask CDH lists.
On Feb 19, 2013, at 12:03 AM, jing wang wrote:
> Hi User,
>
> We're facing the challenge of which hadoop version to choose.
> We prefer to CDH4, but have a few qustions:
> 1.Are MRV1 and MRV2 sharing the same hdfs? If so, can MRV1 upgrade to MRV2 smoothly?
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
> 3.Is MRV1 stable enough to be used in production?
>
>
> Thanks & best regards,
> Jing Wang
>
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: CDH4 version questions
Posted by Arun C Murthy <ac...@hortonworks.com>.
Pls ask CDH lists.
On Feb 19, 2013, at 12:03 AM, jing wang wrote:
> Hi User,
>
> We're facing the challenge of which hadoop version to choose.
> We prefer to CDH4, but have a few qustions:
> 1.Are MRV1 and MRV2 sharing the same hdfs? If so, can MRV1 upgrade to MRV2 smoothly?
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
> 3.Is MRV1 stable enough to be used in production?
>
>
> Thanks & best regards,
> Jing Wang
>
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: CDH4 version questions
Posted by David Boyd <db...@lorenzresearch.com>.
Jing:
We are using CDH4.1.1 and CDH3 and the
base Apache Hadoop on several different efforts.
My best cut at answering your questions is below
however the CDH lists will likely have better
information on some of them.
On 2/19/2013 3:03 AM, jing wang wrote:
> Hi User,
>
> We're facing the challenge of which hadoop version to choose.
> We prefer to CDH4, but have a few qustions:
> 1.Are MRV1 and MRV2 sharing the same hdfs?
YES.
If so, can MRV1 upgrade
> to MRV2 smoothly?
They should but we have not played with MRV2 much yet.
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
Some. You will certainly get deprecated warnings. Biggest issue
seems to be getting the right CDH4 libraries in the path.
> 3.Is MRV1 stable enough to be used in production?
Yes. I have been beating on a CDH4 with MRV1 code heavily.
This includes running the SWIM benchmark with the Facebook
workload.
Even cloudera does not thing MRV2 is quite stable enough yet.
>
>
> Thanks & best regards,
> Jing Wang
>
>
>
--
========= mailto:dboyd@lorenzresearch.com ============
David W. Boyd
Vice President, Operations
Lorenz Research, a Data Tactics corporation
7901 Jones Branch, Suite 610
Mclean, VA 22102
office: +1-703-506-3735, ext 308
fax: +1-703-506-6703
cell: +1-703-402-7908
============== http://www.lorenzresearch.com/ ============
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.
Re: CDH4 version questions
Posted by David Boyd <db...@lorenzresearch.com>.
Jing:
We are using CDH4.1.1 and CDH3 and the
base Apache Hadoop on several different efforts.
My best cut at answering your questions is below
however the CDH lists will likely have better
information on some of them.
On 2/19/2013 3:03 AM, jing wang wrote:
> Hi User,
>
> We're facing the challenge of which hadoop version to choose.
> We prefer to CDH4, but have a few qustions:
> 1.Are MRV1 and MRV2 sharing the same hdfs?
YES.
If so, can MRV1 upgrade
> to MRV2 smoothly?
They should but we have not played with MRV2 much yet.
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
Some. You will certainly get deprecated warnings. Biggest issue
seems to be getting the right CDH4 libraries in the path.
> 3.Is MRV1 stable enough to be used in production?
Yes. I have been beating on a CDH4 with MRV1 code heavily.
This includes running the SWIM benchmark with the Facebook
workload.
Even cloudera does not thing MRV2 is quite stable enough yet.
>
>
> Thanks & best regards,
> Jing Wang
>
>
>
--
========= mailto:dboyd@lorenzresearch.com ============
David W. Boyd
Vice President, Operations
Lorenz Research, a Data Tactics corporation
7901 Jones Branch, Suite 610
Mclean, VA 22102
office: +1-703-506-3735, ext 308
fax: +1-703-506-6703
cell: +1-703-402-7908
============== http://www.lorenzresearch.com/ ============
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.
Re: CDH4 version questions
Posted by David Boyd <db...@lorenzresearch.com>.
Jing:
We are using CDH4.1.1 and CDH3 and the
base Apache Hadoop on several different efforts.
My best cut at answering your questions is below
however the CDH lists will likely have better
information on some of them.
On 2/19/2013 3:03 AM, jing wang wrote:
> Hi User,
>
> We're facing the challenge of which hadoop version to choose.
> We prefer to CDH4, but have a few qustions:
> 1.Are MRV1 and MRV2 sharing the same hdfs?
YES.
If so, can MRV1 upgrade
> to MRV2 smoothly?
They should but we have not played with MRV2 much yet.
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
Some. You will certainly get deprecated warnings. Biggest issue
seems to be getting the right CDH4 libraries in the path.
> 3.Is MRV1 stable enough to be used in production?
Yes. I have been beating on a CDH4 with MRV1 code heavily.
This includes running the SWIM benchmark with the Facebook
workload.
Even cloudera does not thing MRV2 is quite stable enough yet.
>
>
> Thanks & best regards,
> Jing Wang
>
>
>
--
========= mailto:dboyd@lorenzresearch.com ============
David W. Boyd
Vice President, Operations
Lorenz Research, a Data Tactics corporation
7901 Jones Branch, Suite 610
Mclean, VA 22102
office: +1-703-506-3735, ext 308
fax: +1-703-506-6703
cell: +1-703-402-7908
============== http://www.lorenzresearch.com/ ============
The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.
Re: CDH4 version questions
Posted by Arun C Murthy <ac...@hortonworks.com>.
Pls ask CDH lists.
On Feb 19, 2013, at 12:03 AM, jing wang wrote:
> Hi User,
>
> We're facing the challenge of which hadoop version to choose.
> We prefer to CDH4, but have a few qustions:
> 1.Are MRV1 and MRV2 sharing the same hdfs? If so, can MRV1 upgrade to MRV2 smoothly?
> 2.If using MRV1, should our m/r code basing CDH3 be changed?
> 3.Is MRV1 stable enough to be used in production?
>
>
> Thanks & best regards,
> Jing Wang
>
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/