You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Nirmal Kumar <ni...@impetus.co.in> on 2014/01/16 13:14:20 UTC
Hadoop migration downtime: Hadoop 1.x to 2.x
All,
I want to know while upgrading\migrating from Apache Hadoop 1.x to 2.x(MRv2+YARN) in a production cluster of several nodes is there any *anticipated downtime* that one needs to be aware of?
I am asking this since this will hamper the *HA (High Availability)*.
IMHO as with any other upgrade\migrate process, it will have some *manual steps*(sanity checks, backups) and need *thorough testing* before going live.
Please share your opinions on any other reasons for the downtime.
Thanks,
-Nirmal
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
RE: Hadoop migration downtime: Hadoop 1.x to 2.x
Posted by Nirmal Kumar <ni...@impetus.co.in>.
Hi All,
Any thoughts to share?
-Nirmal
From: Nirmal Kumar
Sent: Thursday, January 16, 2014 5:44 PM
To: user@hadoop.apache.org
Subject: Hadoop migration downtime: Hadoop 1.x to 2.x
All,
I want to know while upgrading\migrating from Apache Hadoop 1.x to 2.x(MRv2+YARN) in a production cluster of several nodes is there any *anticipated downtime* that one needs to be aware of?
I am asking this since this will hamper the *HA (High Availability)*.
IMHO as with any other upgrade\migrate process, it will have some *manual steps*(sanity checks, backups) and need *thorough testing* before going live.
Please share your opinions on any other reasons for the downtime.
Thanks,
-Nirmal
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
RE: Hadoop migration downtime: Hadoop 1.x to 2.x
Posted by Nirmal Kumar <ni...@impetus.co.in>.
Hi All,
Any thoughts to share?
-Nirmal
From: Nirmal Kumar
Sent: Thursday, January 16, 2014 5:44 PM
To: user@hadoop.apache.org
Subject: Hadoop migration downtime: Hadoop 1.x to 2.x
All,
I want to know while upgrading\migrating from Apache Hadoop 1.x to 2.x(MRv2+YARN) in a production cluster of several nodes is there any *anticipated downtime* that one needs to be aware of?
I am asking this since this will hamper the *HA (High Availability)*.
IMHO as with any other upgrade\migrate process, it will have some *manual steps*(sanity checks, backups) and need *thorough testing* before going live.
Please share your opinions on any other reasons for the downtime.
Thanks,
-Nirmal
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Hadoop migration downtime: Hadoop 1.x to 2.x
Posted by Ravi Mutyala <ra...@hortonworks.com>.
If you have a mirror of your production environment (not in size but in
versions of hardware and software), I suggest you run migration steps on
this environment to test and measure the down times.
Actual down times are dependent on components in use. You could also do
some of your backup steps by pushing namenode to safemode (which is still
down time of your cluster, but you could still be able to do some readonly
operations on the cluster)
That said, we migrated some large clusters in couple of hours but spent
days upgrading and rolling back test environments before production
environments. Also consider automating what ever you can when working of
the test environment.
Ravi
On Thu, Jan 16, 2014 at 6:14 AM, Nirmal Kumar <ni...@impetus.co.in>wrote:
> All,
>
>
>
> I want to know while upgrading\migrating from Apache Hadoop 1.x to
> 2.x(MRv2+YARN) in a production cluster of several nodes is there any **anticipated
> downtime** that one needs to be aware of?
>
> I am asking this since this will hamper the **HA (High Availability)**.
>
>
>
> IMHO as with any other upgrade\migrate process, it will have some **manual
> steps**(sanity checks, backups) and need **thorough testing** before
> going live.
>
>
>
>
> Please share your opinions on any other reasons for the downtime.
>
>
>
> Thanks,
>
> -Nirmal
>
> ------------------------------
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Hadoop migration downtime: Hadoop 1.x to 2.x
Posted by Ravi Mutyala <ra...@hortonworks.com>.
If you have a mirror of your production environment (not in size but in
versions of hardware and software), I suggest you run migration steps on
this environment to test and measure the down times.
Actual down times are dependent on components in use. You could also do
some of your backup steps by pushing namenode to safemode (which is still
down time of your cluster, but you could still be able to do some readonly
operations on the cluster)
That said, we migrated some large clusters in couple of hours but spent
days upgrading and rolling back test environments before production
environments. Also consider automating what ever you can when working of
the test environment.
Ravi
On Thu, Jan 16, 2014 at 6:14 AM, Nirmal Kumar <ni...@impetus.co.in>wrote:
> All,
>
>
>
> I want to know while upgrading\migrating from Apache Hadoop 1.x to
> 2.x(MRv2+YARN) in a production cluster of several nodes is there any **anticipated
> downtime** that one needs to be aware of?
>
> I am asking this since this will hamper the **HA (High Availability)**.
>
>
>
> IMHO as with any other upgrade\migrate process, it will have some **manual
> steps**(sanity checks, backups) and need **thorough testing** before
> going live.
>
>
>
>
> Please share your opinions on any other reasons for the downtime.
>
>
>
> Thanks,
>
> -Nirmal
>
> ------------------------------
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: Hadoop migration downtime: Hadoop 1.x to 2.x
Posted by Nirmal Kumar <ni...@impetus.co.in>.
Hi All,
Any thoughts to share?
-Nirmal
From: Nirmal Kumar
Sent: Thursday, January 16, 2014 5:44 PM
To: user@hadoop.apache.org
Subject: Hadoop migration downtime: Hadoop 1.x to 2.x
All,
I want to know while upgrading\migrating from Apache Hadoop 1.x to 2.x(MRv2+YARN) in a production cluster of several nodes is there any *anticipated downtime* that one needs to be aware of?
I am asking this since this will hamper the *HA (High Availability)*.
IMHO as with any other upgrade\migrate process, it will have some *manual steps*(sanity checks, backups) and need *thorough testing* before going live.
Please share your opinions on any other reasons for the downtime.
Thanks,
-Nirmal
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
Re: Hadoop migration downtime: Hadoop 1.x to 2.x
Posted by Ravi Mutyala <ra...@hortonworks.com>.
If you have a mirror of your production environment (not in size but in
versions of hardware and software), I suggest you run migration steps on
this environment to test and measure the down times.
Actual down times are dependent on components in use. You could also do
some of your backup steps by pushing namenode to safemode (which is still
down time of your cluster, but you could still be able to do some readonly
operations on the cluster)
That said, we migrated some large clusters in couple of hours but spent
days upgrading and rolling back test environments before production
environments. Also consider automating what ever you can when working of
the test environment.
Ravi
On Thu, Jan 16, 2014 at 6:14 AM, Nirmal Kumar <ni...@impetus.co.in>wrote:
> All,
>
>
>
> I want to know while upgrading\migrating from Apache Hadoop 1.x to
> 2.x(MRv2+YARN) in a production cluster of several nodes is there any **anticipated
> downtime** that one needs to be aware of?
>
> I am asking this since this will hamper the **HA (High Availability)**.
>
>
>
> IMHO as with any other upgrade\migrate process, it will have some **manual
> steps**(sanity checks, backups) and need **thorough testing** before
> going live.
>
>
>
>
> Please share your opinions on any other reasons for the downtime.
>
>
>
> Thanks,
>
> -Nirmal
>
> ------------------------------
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Hadoop migration downtime: Hadoop 1.x to 2.x
Posted by Ravi Mutyala <ra...@hortonworks.com>.
If you have a mirror of your production environment (not in size but in
versions of hardware and software), I suggest you run migration steps on
this environment to test and measure the down times.
Actual down times are dependent on components in use. You could also do
some of your backup steps by pushing namenode to safemode (which is still
down time of your cluster, but you could still be able to do some readonly
operations on the cluster)
That said, we migrated some large clusters in couple of hours but spent
days upgrading and rolling back test environments before production
environments. Also consider automating what ever you can when working of
the test environment.
Ravi
On Thu, Jan 16, 2014 at 6:14 AM, Nirmal Kumar <ni...@impetus.co.in>wrote:
> All,
>
>
>
> I want to know while upgrading\migrating from Apache Hadoop 1.x to
> 2.x(MRv2+YARN) in a production cluster of several nodes is there any **anticipated
> downtime** that one needs to be aware of?
>
> I am asking this since this will hamper the **HA (High Availability)**.
>
>
>
> IMHO as with any other upgrade\migrate process, it will have some **manual
> steps**(sanity checks, backups) and need **thorough testing** before
> going live.
>
>
>
>
> Please share your opinions on any other reasons for the downtime.
>
>
>
> Thanks,
>
> -Nirmal
>
> ------------------------------
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: Hadoop migration downtime: Hadoop 1.x to 2.x
Posted by Nirmal Kumar <ni...@impetus.co.in>.
Hi All,
Any thoughts to share?
-Nirmal
From: Nirmal Kumar
Sent: Thursday, January 16, 2014 5:44 PM
To: user@hadoop.apache.org
Subject: Hadoop migration downtime: Hadoop 1.x to 2.x
All,
I want to know while upgrading\migrating from Apache Hadoop 1.x to 2.x(MRv2+YARN) in a production cluster of several nodes is there any *anticipated downtime* that one needs to be aware of?
I am asking this since this will hamper the *HA (High Availability)*.
IMHO as with any other upgrade\migrate process, it will have some *manual steps*(sanity checks, backups) and need *thorough testing* before going live.
Please share your opinions on any other reasons for the downtime.
Thanks,
-Nirmal
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.