You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Tsz Wo Sze <sz...@yahoo.com> on 2014/02/25 22:41:44 UTC
[VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Hi hdfs-dev,
We propose merging the HDFS-5535 branch to trunk.
HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode services and wire compatibility. These two capabilities make it feasible to upgrade HDFS without incurring HDFS downtime. We make such improvement in the HDFS-5535 branch.
The HDFS-5535 branch is ready to be merged to trunk. As this being written, there are 48 subtasks in HDFS-5535; 44 subtasks are already completed. The core developments including feature development, unit tests and user doc, are already done. The merge patch posted a few ago already passed Jenkins. I will post a updated patch to trigger Jenkins again for the latest code base.
The remaining JIRAs are:
HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable rolling upgrades (assigned to Sanjay)
HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned to Jing)
HDFS-6013: add rollingUpgrade information to latest UI (assigned to Vinay)
HDFS-6016: Update datanode replacement policy to make writes more robust (assigned to Kihwal)
HDFS-6000 will be committed soon. All other issues are further improvements which can be done after merge.
The other remaining works are:
- Revise the design doc
- Post a test plan (Haohui is working on it.)
- Execute the manual tests (Haohui and Fengdong will work on it.)
The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai, Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have proposed the design, worked on the code, reviewed patches, tested the features and authored documentation. We thank everyone that who has gave us valuable comments and feedback on the feature.
The vote runs for 7 days. Here is my +1 on the merge.
Thanks.
Tsz-Wo
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Tsz Wo Sze <sz...@yahoo.com>.
Hi hdfs-dev,
With ten +1's (eight binding and two non-binding), the vote passed. Once the latest merge patch has passed Jenkins, I will merge the HDFS-5535 branch to trunk. Thanks everyone!
Tsz-Wo
On Tuesday, February 25, 2014 1:41 PM, Tsz Wo Sze <sz...@yahoo.com> wrote:
Hi hdfs-dev,
>
>
>We propose merging the HDFS-5535 branch to trunk.
>
>
>HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode services and wire compatibility. These two capabilities make it feasible to upgrade HDFS without incurring HDFS downtime. We make such improvement in the HDFS-5535 branch.
>
>
>The HDFS-5535 branch is ready to be merged to trunk. As this being written, there are 48 subtasks in HDFS-5535; 44 subtasks are already completed. The core developments including feature development, unit tests and user doc, are already done. The merge patch posted a few ago already passed Jenkins. I will post a updated patch to trigger Jenkins again for the latest code base.
>
>
>The remaining JIRAs are:
>
>
>HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable rolling upgrades (assigned to Sanjay)
>HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned to Jing)
>HDFS-6013: add rollingUpgrade information to latest UI (assigned to Vinay)
>HDFS-6016: Update datanode replacement policy to make writes more robust (assigned to Kihwal)
>
>
>HDFS-6000 will be committed soon. All other issues are further improvements which can be done after merge.
>
>
>The other remaining works are:
>- Revise the design doc
>- Post a test plan (Haohui is working on it.)
>- Execute the manual tests (Haohui and Fengdong will work on it.)
>
>
>The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai, Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have proposed the design, worked on the code, reviewed patches, tested the features and authored documentation. We thank everyone that who has gave us valuable comments and feedback on the feature.
>
>
>The vote runs for 7 days. Here is my +1 on the merge.
>
>
>Thanks.
>Tsz-Wo
>
>
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Tsz Wo Sze <sz...@yahoo.com>.
Hi Andrew,
Suresh has posted a new revision of the design doc. Have you got a chance to take a look? Kihwal is also revising the design doc for the datanode sections.
Thanks for voting!
Tsz-Wo
On Monday, March 3, 2014 10:57 AM, Andrew Wang <an...@cloudera.com> wrote:
Can we get an updated design doc posted to the JIRA? I'm not -1 or
>anything, but I asked for this on the JIRA just before the merge vote
>started, and would like to see it posted soon.
>
>
>
>On Mon, Mar 3, 2014 at 9:59 AM, Arpit Agarwal <aa...@hortonworks.com>wrote:
>
>> +1 for merging the feature to trunk.
>>
>>
>> On Tue, Feb 25, 2014 at 1:41 PM, Tsz Wo Sze <sz...@yahoo.com> wrote:
>>
>> > Hi hdfs-dev,
>> >
>> > We propose merging the HDFS-5535 branch to trunk.
>> >
>> > HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
>> > daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
>> > services and wire compatibility. These two capabilities make it feasible
>> to
>> > upgrade HDFS without incurring HDFS downtime. We make such improvement
>> in
>> > the HDFS-5535 branch.
>> >
>> > The HDFS-5535 branch is ready to be merged to trunk. As this being
>> > written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
>> > completed. The core developments including feature development, unit
>> tests
>> > and user doc, are already done. The merge patch posted a few ago already
>> > passed Jenkins. I will post a updated patch to trigger Jenkins again for
>> > the latest code base.
>> >
>> > The remaining JIRAs are:
>> >
>> > HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable
>> rolling
>> > upgrades (assigned to Sanjay)
>> > HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned
>> > to Jing)
>> > HDFS-6013: add rollingUpgrade information to latest UI (assigned to
>> Vinay)
>> > HDFS-6016: Update datanode replacement policy to make writes more robust
>> > (assigned to Kihwal)
>> >
>> > HDFS-6000 will be committed soon. All other issues are further
>> > improvements which can be done after merge.
>> >
>> > The other remaining works are:
>> > - Revise the design doc
>> > - Post a test plan (Haohui is working on it.)
>> > - Execute the manual tests (Haohui and Fengdong will work on it.)
>> >
>> > The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh
>> > Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai,
>> > Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who
>> have
>> > proposed the design, worked on the code, reviewed patches, tested the
>> > features and authored documentation. We thank everyone that who has gave
>> > us valuable comments and feedback on the feature.
>> >
>> > The vote runs for 7 days. Here is my +1 on the merge.
>> >
>> > Thanks.
>> > Tsz-Wo
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>
>
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Andrew Wang <an...@cloudera.com>.
Can we get an updated design doc posted to the JIRA? I'm not -1 or
anything, but I asked for this on the JIRA just before the merge vote
started, and would like to see it posted soon.
On Mon, Mar 3, 2014 at 9:59 AM, Arpit Agarwal <aa...@hortonworks.com>wrote:
> +1 for merging the feature to trunk.
>
>
> On Tue, Feb 25, 2014 at 1:41 PM, Tsz Wo Sze <sz...@yahoo.com> wrote:
>
> > Hi hdfs-dev,
> >
> > We propose merging the HDFS-5535 branch to trunk.
> >
> > HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
> > daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
> > services and wire compatibility. These two capabilities make it feasible
> to
> > upgrade HDFS without incurring HDFS downtime. We make such improvement
> in
> > the HDFS-5535 branch.
> >
> > The HDFS-5535 branch is ready to be merged to trunk. As this being
> > written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
> > completed. The core developments including feature development, unit
> tests
> > and user doc, are already done. The merge patch posted a few ago already
> > passed Jenkins. I will post a updated patch to trigger Jenkins again for
> > the latest code base.
> >
> > The remaining JIRAs are:
> >
> > HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable
> rolling
> > upgrades (assigned to Sanjay)
> > HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned
> > to Jing)
> > HDFS-6013: add rollingUpgrade information to latest UI (assigned to
> Vinay)
> > HDFS-6016: Update datanode replacement policy to make writes more robust
> > (assigned to Kihwal)
> >
> > HDFS-6000 will be committed soon. All other issues are further
> > improvements which can be done after merge.
> >
> > The other remaining works are:
> > - Revise the design doc
> > - Post a test plan (Haohui is working on it.)
> > - Execute the manual tests (Haohui and Fengdong will work on it.)
> >
> > The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh
> > Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai,
> > Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who
> have
> > proposed the design, worked on the code, reviewed patches, tested the
> > features and authored documentation. We thank everyone that who has gave
> > us valuable comments and feedback on the feature.
> >
> > The vote runs for 7 days. Here is my +1 on the merge.
> >
> > Thanks.
> > Tsz-Wo
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Arpit Agarwal <aa...@hortonworks.com>.
+1 for merging the feature to trunk.
On Tue, Feb 25, 2014 at 1:41 PM, Tsz Wo Sze <sz...@yahoo.com> wrote:
> Hi hdfs-dev,
>
> We propose merging the HDFS-5535 branch to trunk.
>
> HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
> daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
> services and wire compatibility. These two capabilities make it feasible to
> upgrade HDFS without incurring HDFS downtime. We make such improvement in
> the HDFS-5535 branch.
>
> The HDFS-5535 branch is ready to be merged to trunk. As this being
> written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
> completed. The core developments including feature development, unit tests
> and user doc, are already done. The merge patch posted a few ago already
> passed Jenkins. I will post a updated patch to trigger Jenkins again for
> the latest code base.
>
> The remaining JIRAs are:
>
> HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable rolling
> upgrades (assigned to Sanjay)
> HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned
> to Jing)
> HDFS-6013: add rollingUpgrade information to latest UI (assigned to Vinay)
> HDFS-6016: Update datanode replacement policy to make writes more robust
> (assigned to Kihwal)
>
> HDFS-6000 will be committed soon. All other issues are further
> improvements which can be done after merge.
>
> The other remaining works are:
> - Revise the design doc
> - Post a test plan (Haohui is working on it.)
> - Execute the manual tests (Haohui and Fengdong will work on it.)
>
> The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh
> Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai,
> Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have
> proposed the design, worked on the code, reviewed patches, tested the
> features and authored documentation. We thank everyone that who has gave
> us valuable comments and feedback on the feature.
>
> The vote runs for 7 days. Here is my +1 on the merge.
>
> Thanks.
> Tsz-Wo
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Jun Ping Du <jd...@vmware.com>.
+1 for merging. IMO, Rolling Upgrade is a significant feature for 0 downtime HDFS. The reminding work looks reasonable to me. Thanks Nicholas and all for hard working on this!
CC to yarn-dev, look forward to see YARN Rolling Upgrade can onboard soon (YARN-666).
Cheers,
Junping
----- Original Message -----
From: "Tsz Wo Sze" <sz...@yahoo.com>
To: "hdfs dev" <hd...@hadoop.apache.org>
Sent: Wednesday, February 26, 2014 5:41:44 AM
Subject: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Hi hdfs-dev,
We propose merging the HDFS-5535 branch to trunk.
HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode services and wire compatibility. These two capabilities make it feasible to upgrade HDFS without incurring HDFS downtime. We make such improvement in the HDFS-5535 branch.
The HDFS-5535 branch is ready to be merged to trunk. As this being written, there are 48 subtasks in HDFS-5535; 44 subtasks are already completed. The core developments including feature development, unit tests and user doc, are already done. The merge patch posted a few ago already passed Jenkins. I will post a updated patch to trigger Jenkins again for the latest code base.
The remaining JIRAs are:
HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable rolling upgrades (assigned to Sanjay)
HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned to Jing)
HDFS-6013: add rollingUpgrade information to latest UI (assigned to Vinay)
HDFS-6016: Update datanode replacement policy to make writes more robust (assigned to Kihwal)
HDFS-6000 will be committed soon. All other issues are further improvements which can be done after merge.
The other remaining works are:
- Revise the design doc
- Post a test plan (Haohui is working on it.)
- Execute the manual tests (Haohui and Fengdong will work on it.)
The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai, Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have proposed the design, worked on the code, reviewed patches, tested the features and authored documentation. We thank everyone that who has gave us valuable comments and feedback on the feature.
The vote runs for 7 days. Here is my +1 on the merge.
Thanks.
Tsz-Wo
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Suresh Srinivas <su...@hortonworks.com>.
+1 for merging the feature to trunk.
Main goal of HDFS-3225 was rolling upgrade. I have resolved it, given the
umbrella jira addresses it.
On Tue, Feb 25, 2014 at 1:41 PM, Tsz Wo Sze <sz...@yahoo.com> wrote:
> Hi hdfs-dev,
>
> We propose merging the HDFS-5535 branch to trunk.
>
> HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
> daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
> services and wire compatibility. These two capabilities make it feasible to
> upgrade HDFS without incurring HDFS downtime. We make such improvement in
> the HDFS-5535 branch.
>
> The HDFS-5535 branch is ready to be merged to trunk. As this being
> written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
> completed. The core developments including feature development, unit tests
> and user doc, are already done. The merge patch posted a few ago already
> passed Jenkins. I will post a updated patch to trigger Jenkins again for
> the latest code base.
>
> The remaining JIRAs are:
>
> HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable rolling
> upgrades (assigned to Sanjay)
> HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned
> to Jing)
> HDFS-6013: add rollingUpgrade information to latest UI (assigned to Vinay)
> HDFS-6016: Update datanode replacement policy to make writes more robust
> (assigned to Kihwal)
>
> HDFS-6000 will be committed soon. All other issues are further
> improvements which can be done after merge.
>
> The other remaining works are:
> - Revise the design doc
> - Post a test plan (Haohui is working on it.)
> - Execute the manual tests (Haohui and Fengdong will work on it.)
>
> The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh
> Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai,
> Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have
> proposed the design, worked on the code, reviewed patches, tested the
> features and authored documentation. We thank everyone that who has gave
> us valuable comments and feedback on the feature.
>
> The vote runs for 7 days. Here is my +1 on the merge.
>
> Thanks.
> Tsz-Wo
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Jun Ping Du <jd...@vmware.com>.
+1 for merging. IMO, Rolling Upgrade is a significant feature for 0 downtime HDFS. The reminding work looks reasonable to me. Thanks Nicholas and all for hard working on this!
CC to yarn-dev, look forward to see YARN Rolling Upgrade can onboard soon (YARN-666).
Cheers,
Junping
----- Original Message -----
From: "Tsz Wo Sze" <sz...@yahoo.com>
To: "hdfs dev" <hd...@hadoop.apache.org>
Sent: Wednesday, February 26, 2014 5:41:44 AM
Subject: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Hi hdfs-dev,
We propose merging the HDFS-5535 branch to trunk.
HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode services and wire compatibility. These two capabilities make it feasible to upgrade HDFS without incurring HDFS downtime. We make such improvement in the HDFS-5535 branch.
The HDFS-5535 branch is ready to be merged to trunk. As this being written, there are 48 subtasks in HDFS-5535; 44 subtasks are already completed. The core developments including feature development, unit tests and user doc, are already done. The merge patch posted a few ago already passed Jenkins. I will post a updated patch to trigger Jenkins again for the latest code base.
The remaining JIRAs are:
HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable rolling upgrades (assigned to Sanjay)
HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned to Jing)
HDFS-6013: add rollingUpgrade information to latest UI (assigned to Vinay)
HDFS-6016: Update datanode replacement policy to make writes more robust (assigned to Kihwal)
HDFS-6000 will be committed soon. All other issues are further improvements which can be done after merge.
The other remaining works are:
- Revise the design doc
- Post a test plan (Haohui is working on it.)
- Execute the manual tests (Haohui and Fengdong will work on it.)
The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai, Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have proposed the design, worked on the code, reviewed patches, tested the features and authored documentation. We thank everyone that who has gave us valuable comments and feedback on the feature.
The vote runs for 7 days. Here is my +1 on the merge.
Thanks.
Tsz-Wo
RE: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Vinayakumar B <vi...@huawei.com>.
+1 for merging.
-----Original Message-----
From: Brandon Li [mailto:brandon@hortonworks.com]
Sent: 26 February 2014 05:42
To: hdfs-dev@hadoop.apache.org
Subject: Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
+1 for merging it to trunk.
On Tue, Feb 25, 2014 at 3:23 PM, Azuryy Yu <az...@gmail.com> wrote:
> +1 for merging.
> On Feb 26, 2014 5:42 AM, "Tsz Wo Sze" <sz...@yahoo.com> wrote:
>
> > Hi hdfs-dev,
> >
> > We propose merging the HDFS-5535 branch to trunk.
> >
> > HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
> > daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
> > services and wire compatibility. These two capabilities make it
> > feasible
> to
> > upgrade HDFS without incurring HDFS downtime. We make such
> > improvement
> in
> > the HDFS-5535 branch.
> >
> > The HDFS-5535 branch is ready to be merged to trunk. As this being
> > written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
> > completed. The core developments including feature development,
> > unit
> tests
> > and user doc, are already done. The merge patch posted a few ago
> > already passed Jenkins. I will post a updated patch to trigger
> > Jenkins again for the latest code base.
> >
> > The remaining JIRAs are:
> >
> > HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable
> rolling
> > upgrades (assigned to Sanjay)
> > HDFS-6000: Avoid saving namespace when starting rolling upgrade
> > (assigned to Jing)
> > HDFS-6013: add rollingUpgrade information to latest UI (assigned to
> Vinay)
> > HDFS-6016: Update datanode replacement policy to make writes more
> > robust (assigned to Kihwal)
> >
> > HDFS-6000 will be committed soon. All other issues are further
> > improvements which can be done after merge.
> >
> > The other remaining works are:
> > - Revise the design doc
> > - Post a test plan (Haohui is working on it.)
> > - Execute the manual tests (Haohui and Fengdong will work on it.)
> >
> > The work was a collective effort of Nathan Roberts, Sanjay Radia,
> > Suresh Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li,
> > Haohui Mai, Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo
> > Nicholas Sze, who
> have
> > proposed the design, worked on the code, reviewed patches, tested
> > the features and authored documentation. We thank everyone that who
> > has gave us valuable comments and feedback on the feature.
> >
> > The vote runs for 7 days. Here is my +1 on the merge.
> >
> > Thanks.
> > Tsz-Wo
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Brandon Li <br...@hortonworks.com>.
+1 for merging it to trunk.
On Tue, Feb 25, 2014 at 3:23 PM, Azuryy Yu <az...@gmail.com> wrote:
> +1 for merging.
> On Feb 26, 2014 5:42 AM, "Tsz Wo Sze" <sz...@yahoo.com> wrote:
>
> > Hi hdfs-dev,
> >
> > We propose merging the HDFS-5535 branch to trunk.
> >
> > HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
> > daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
> > services and wire compatibility. These two capabilities make it feasible
> to
> > upgrade HDFS without incurring HDFS downtime. We make such improvement
> in
> > the HDFS-5535 branch.
> >
> > The HDFS-5535 branch is ready to be merged to trunk. As this being
> > written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
> > completed. The core developments including feature development, unit
> tests
> > and user doc, are already done. The merge patch posted a few ago already
> > passed Jenkins. I will post a updated patch to trigger Jenkins again for
> > the latest code base.
> >
> > The remaining JIRAs are:
> >
> > HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable
> rolling
> > upgrades (assigned to Sanjay)
> > HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned
> > to Jing)
> > HDFS-6013: add rollingUpgrade information to latest UI (assigned to
> Vinay)
> > HDFS-6016: Update datanode replacement policy to make writes more robust
> > (assigned to Kihwal)
> >
> > HDFS-6000 will be committed soon. All other issues are further
> > improvements which can be done after merge.
> >
> > The other remaining works are:
> > - Revise the design doc
> > - Post a test plan (Haohui is working on it.)
> > - Execute the manual tests (Haohui and Fengdong will work on it.)
> >
> > The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh
> > Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai,
> > Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who
> have
> > proposed the design, worked on the code, reviewed patches, tested the
> > features and authored documentation. We thank everyone that who has gave
> > us valuable comments and feedback on the feature.
> >
> > The vote runs for 7 days. Here is my +1 on the merge.
> >
> > Thanks.
> > Tsz-Wo
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Azuryy Yu <az...@gmail.com>.
+1 for merging.
On Feb 26, 2014 5:42 AM, "Tsz Wo Sze" <sz...@yahoo.com> wrote:
> Hi hdfs-dev,
>
> We propose merging the HDFS-5535 branch to trunk.
>
> HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
> daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
> services and wire compatibility. These two capabilities make it feasible to
> upgrade HDFS without incurring HDFS downtime. We make such improvement in
> the HDFS-5535 branch.
>
> The HDFS-5535 branch is ready to be merged to trunk. As this being
> written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
> completed. The core developments including feature development, unit tests
> and user doc, are already done. The merge patch posted a few ago already
> passed Jenkins. I will post a updated patch to trigger Jenkins again for
> the latest code base.
>
> The remaining JIRAs are:
>
> HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable rolling
> upgrades (assigned to Sanjay)
> HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned
> to Jing)
> HDFS-6013: add rollingUpgrade information to latest UI (assigned to Vinay)
> HDFS-6016: Update datanode replacement policy to make writes more robust
> (assigned to Kihwal)
>
> HDFS-6000 will be committed soon. All other issues are further
> improvements which can be done after merge.
>
> The other remaining works are:
> - Revise the design doc
> - Post a test plan (Haohui is working on it.)
> - Execute the manual tests (Haohui and Fengdong will work on it.)
>
> The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh
> Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai,
> Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have
> proposed the design, worked on the code, reviewed patches, tested the
> features and authored documentation. We thank everyone that who has gave
> us valuable comments and feedback on the feature.
>
> The vote runs for 7 days. Here is my +1 on the merge.
>
> Thanks.
> Tsz-Wo
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Jing Zhao <ji...@hortonworks.com>.
+1 for the merging.
On Fri, Feb 28, 2014 at 10:58 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
> +1 for the merge.
>
> I just got caught up on the current state of the branch, and it looks good.
> End user documentation is in place. I deployed a cluster built from the
> branch, and then I used the documentation to test various scenarios of
> rolling upgrade, downgrade and rollback. Everything worked as expected.
> This looks ready to merge.
>
> Nice work, everyone!
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Thu, Feb 27, 2014 at 7:02 PM, Kihwal Lee <ki...@yahoo-inc.com> wrote:
>
> > +1
> >
> > > On Feb 25, 2014, at 3:42 PM, "Tsz Wo Sze" <sz...@yahoo.com> wrote:
> > >
> > > Hi hdfs-dev,
> > >
> > > We propose merging the HDFS-5535 branch to trunk.
> > >
> > > HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
> > daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
> > services and wire compatibility. These two capabilities make it feasible
> to
> > upgrade HDFS without incurring HDFS downtime. We make such improvement
> in
> > the HDFS-5535 branch.
> > >
> > > The HDFS-5535 branch is ready to be merged to trunk. As this being
> > written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
> > completed. The core developments including feature development, unit
> tests
> > and user doc, are already done. The merge patch posted a few ago already
> > passed Jenkins. I will post a updated patch to trigger Jenkins again for
> > the latest code base.
> > >
> > > The remaining JIRAs are:
> > >
> > > HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable
> > rolling upgrades (assigned to Sanjay)
> > > HDFS-6000: Avoid saving namespace when starting rolling upgrade
> > (assigned to Jing)
> > > HDFS-6013: add rollingUpgrade information to latest UI (assigned to
> > Vinay)
> > > HDFS-6016: Update datanode replacement policy to make writes more
> robust
> > (assigned to Kihwal)
> > >
> > > HDFS-6000 will be committed soon. All other issues are further
> > improvements which can be done after merge.
> > >
> > > The other remaining works are:
> > > - Revise the design doc
> > > - Post a test plan (Haohui is working on it.)
> > > - Execute the manual tests (Haohui and Fengdong will work on it.)
> > >
> > > The work was a collective effort of Nathan Roberts, Sanjay Radia,
> Suresh
> > Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai,
> > Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who
> have
> > proposed the design, worked on the code, reviewed patches, tested the
> > features and authored documentation. We thank everyone that who has gave
> > us valuable comments and feedback on the feature.
> > >
> > > The vote runs for 7 days. Here is my +1 on the merge.
> > >
> > > Thanks.
> > > Tsz-Wo
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Chris Nauroth <cn...@hortonworks.com>.
+1 for the merge.
I just got caught up on the current state of the branch, and it looks good.
End user documentation is in place. I deployed a cluster built from the
branch, and then I used the documentation to test various scenarios of
rolling upgrade, downgrade and rollback. Everything worked as expected.
This looks ready to merge.
Nice work, everyone!
Chris Nauroth
Hortonworks
http://hortonworks.com/
On Thu, Feb 27, 2014 at 7:02 PM, Kihwal Lee <ki...@yahoo-inc.com> wrote:
> +1
>
> > On Feb 25, 2014, at 3:42 PM, "Tsz Wo Sze" <sz...@yahoo.com> wrote:
> >
> > Hi hdfs-dev,
> >
> > We propose merging the HDFS-5535 branch to trunk.
> >
> > HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS
> daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode
> services and wire compatibility. These two capabilities make it feasible to
> upgrade HDFS without incurring HDFS downtime. We make such improvement in
> the HDFS-5535 branch.
> >
> > The HDFS-5535 branch is ready to be merged to trunk. As this being
> written, there are 48 subtasks in HDFS-5535; 44 subtasks are already
> completed. The core developments including feature development, unit tests
> and user doc, are already done. The merge patch posted a few ago already
> passed Jenkins. I will post a updated patch to trigger Jenkins again for
> the latest code base.
> >
> > The remaining JIRAs are:
> >
> > HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable
> rolling upgrades (assigned to Sanjay)
> > HDFS-6000: Avoid saving namespace when starting rolling upgrade
> (assigned to Jing)
> > HDFS-6013: add rollingUpgrade information to latest UI (assigned to
> Vinay)
> > HDFS-6016: Update datanode replacement policy to make writes more robust
> (assigned to Kihwal)
> >
> > HDFS-6000 will be committed soon. All other issues are further
> improvements which can be done after merge.
> >
> > The other remaining works are:
> > - Revise the design doc
> > - Post a test plan (Haohui is working on it.)
> > - Execute the manual tests (Haohui and Fengdong will work on it.)
> >
> > The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh
> Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai,
> Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have
> proposed the design, worked on the code, reviewed patches, tested the
> features and authored documentation. We thank everyone that who has gave
> us valuable comments and feedback on the feature.
> >
> > The vote runs for 7 days. Here is my +1 on the merge.
> >
> > Thanks.
> > Tsz-Wo
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement
Posted by Kihwal Lee <ki...@yahoo-inc.com>.
+1
> On Feb 25, 2014, at 3:42 PM, "Tsz Wo Sze" <sz...@yahoo.com> wrote:
>
> Hi hdfs-dev,
>
> We propose merging the HDFS-5535 branch to trunk.
>
> HDFS Rolling Upgrade is a feature to allow upgrading individual HDFS daemons. In Hadoop v2, HDFS supports highly-available (HA) namenode services and wire compatibility. These two capabilities make it feasible to upgrade HDFS without incurring HDFS downtime. We make such improvement in the HDFS-5535 branch.
>
> The HDFS-5535 branch is ready to be merged to trunk. As this being written, there are 48 subtasks in HDFS-5535; 44 subtasks are already completed. The core developments including feature development, unit tests and user doc, are already done. The merge patch posted a few ago already passed Jenkins. I will post a updated patch to trigger Jenkins again for the latest code base.
>
> The remaining JIRAs are:
>
> HDFS-3225: Revist upgrade snapshots, roll back, finalize to enable rolling upgrades (assigned to Sanjay)
> HDFS-6000: Avoid saving namespace when starting rolling upgrade (assigned to Jing)
> HDFS-6013: add rollingUpgrade information to latest UI (assigned to Vinay)
> HDFS-6016: Update datanode replacement policy to make writes more robust (assigned to Kihwal)
>
> HDFS-6000 will be committed soon. All other issues are further improvements which can be done after merge.
>
> The other remaining works are:
> - Revise the design doc
> - Post a test plan (Haohui is working on it.)
> - Execute the manual tests (Haohui and Fengdong will work on it.)
>
> The work was a collective effort of Nathan Roberts, Sanjay Radia, Suresh Srinivas, Kihwal Lee, Jing Zhao, Arpit Agarwal, Brandon Li, Haohui Mai, Vinayakumar B, Fengdong Yu, Chris Nauroth and Tsz-Wo Nicholas Sze, who have proposed the design, worked on the code, reviewed patches, tested the features and authored documentation. We thank everyone that who has gave us valuable comments and feedback on the feature.
>
> The vote runs for 7 days. Here is my +1 on the merge.
>
> Thanks.
> Tsz-Wo