You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Brahma Reddy Battula <br...@apache.org> on 2019/06/09 13:56:28 UTC
Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Hi All,
Given the positive response to the discussion thread [1], here is the
formal vote thread to merge HDFS-13891 in to trunk.
Summary of code changes:
1. Code changes for this branch are done in the hadoop-hdfs-rbf
subproject, there is no impact to hadoop-hdfs and hadoop-common.
2. Added Security support for RBF
3. Added Missing Client Protocol API's
4. Bug Fixes/ Improvments
The vote will run for 7 days, ending Sat June 15th. I will start this
vote with my +1.
Regards,
Brahma Reddy Battula
1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> Thanks Brahma for starting the thread.
> +1 for merging.
>
> He Xiaoqiao
>
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> wrote:
>
>> Thanks Brahma initiating this.
>> +1(non-binding) for merge.
>>
>> @Uber we have almost all changes specially rbf security in production for
>> a
>> while now without issues.
>>
>> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
>> giovanni.fumarola@gmail.com> wrote:
>>
>> > +1 on merging.
>> >
>> > Thanks Brahma for starting the thread.
>> >
>> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
>> >
>> > > Thank you Brahma for pushing this.
>> > >
>> > > As you mentioned, we have already taken most of the changes into
>> > > production.
>> > > I want to highlight that the main contribution is the addition of
>> > security.
>> > > We have been able to test this at a smaller scale (~500 servers and 4
>> > > subclusters) and the performance is great with our current ZooKeeper
>> > > deployment.
>> > > I would also like to highlight that all the changes are constrained to
>> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
>> > >
>> > > +1 on merging
>> > >
>> > > Inigo
>> > >
>> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks Brahma for starting the discussion.
>> > > > I'm +1 for merging this.
>> > > >
>> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
>> > > > with 2 routers (not in production) and running several tests.
>> > > >
>> > > > Regards,
>> > > > Akira
>> > > >
>> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
>> > brahma@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > Dear Hadoop Developers
>> > > > >
>> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
>> We
>> > > have
>> > > > > been working on this feature from last several months.
>> > > > > This feature work received the contributions from different
>> > companies.
>> > > > All
>> > > > > of the feature development happened smoothly and collaboratively
>> in
>> > > > JIRAs.
>> > > > >
>> > > > > Kindly do take a look at the branch and raise issues/concerns that
>> > need
>> > > > to
>> > > > > be addressed before the merge.
>> > > > >
>> > > > > *Highlights of HDFS-13891 Branch:*
>> > > > > =============================
>> > > > >
>> > > > > Adding Security to RBF(1)
>> > > > > Adding Missing Client API's(2)
>> > > > > Improvements/Bug Fixing
>> > > > > Critical - HDFS-13637, HDFS-13834
>> > > > >
>> > > > > *Commits:*
>> > > > > ========
>> > > > >
>> > > > > No of JIRAs Resolved: 72
>> > > > >
>> > > > > All this commits are in RBF Module. No changes in hdfs/common.
>> > > > >
>> > > > > *Tested Cluster:*
>> > > > > =============
>> > > > >
>> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
>> > other
>> > > > > compaines.
>> > > > >
>> > > > > *Uber*: Most changes are running in production @Uber including the
>> > > > critical
>> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
>> Routers.
>> > > > > Zookeeper as a state store to hold delegation tokens were also
>> stress
>> > > > > tested to hold more than 2 Million tokens. --CR Hota
>> > > > >
>> > > > > *MicroSoft*: Most of these changes are currently running in
>> > production
>> > > at
>> > > > > Microsoft.The security has also been tested in a 500 server
>> cluster
>> > > with
>> > > > 4
>> > > > > subclsuters. --Inigo Goiri
>> > > > >
>> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
>> > > > > routers.Planning deploy 10K production cluster.
>> > > > >
>> > > > > *Contributors:*
>> > > > > ===========
>> > > > >
>> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
>> Asanuma,Shubham
>> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
>> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
>> > > Xiaoqiao,Sherwood
>> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
>> > discussions
>> > > > and
>> > > > > contributing to this.
>> > > > >
>> > > > > *Future Tasks:*
>> > > > > ============
>> > > > >
>> > > > > will cleanup the jira's under this umbrella and contiue to work.
>> > > > >
>> > > > > Reference:
>> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
>> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --Brahma Reddy Battula
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>> Regards,
>> Chitta
>>
>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
Re: [Notice] Merge HDFS-13891(RBF) to trunk
Posted by Brahma Reddy Battula <br...@hotmail.com>.
Merged. thanks to Arpit for quoting the merge process.
________________________________
From: Brahma Reddy Battula <br...@hotmail.com>
Sent: Tuesday, June 25, 2019 12:39 AM
To: hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Cc: Akira Ajisaka; Chittaranjan Hota; Giovanni Matteo Fumarola; Hadoop Common; Hdfs-dev; Iñigo Goiri; Hadoop Common
Subject: [Notice] Merge HDFS-13891(RBF) to trunk
Rebased HDFS-13891 against trunk and ran the jenkins (1).Only two four testcases are failing which flaky and there is already jira to track this. (HDFS-14461)
Going to merge the branch soon which contains 77 commits,so please bear with us.
1.https://builds.apache.org/job/PreCommit-HDFS-Build/27044/console
________________________________
From: Brahma Reddy Battula <br...@apache.org>
Sent: Sunday, June 9, 2019 9:56 PM
To: Xiaoqiao He
Cc: Akira Ajisaka; Chittaranjan Hota; Giovanni Matteo Fumarola; Hadoop Common; Hdfs-dev; Iñigo Goiri
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Hi All,
Given the positive response to the discussion thread [1], here is the
formal vote thread to merge HDFS-13891 in to trunk.
Summary of code changes:
1. Code changes for this branch are done in the hadoop-hdfs-rbf
subproject, there is no impact to hadoop-hdfs and hadoop-common.
2. Added Security support for RBF
3. Added Missing Client Protocol API's
4. Bug Fixes/ Improvments
The vote will run for 7 days, ending Sat June 15th. I will start this
vote with my +1.
Regards,
Brahma Reddy Battula
1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> Thanks Brahma for starting the thread.
> +1 for merging.
>
> He Xiaoqiao
>
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> wrote:
>
>> Thanks Brahma initiating this.
>> +1(non-binding) for merge.
>>
>> @Uber we have almost all changes specially rbf security in production for
>> a
>> while now without issues.
>>
>> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
>> giovanni.fumarola@gmail.com> wrote:
>>
>> > +1 on merging.
>> >
>> > Thanks Brahma for starting the thread.
>> >
>> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
>> >
>> > > Thank you Brahma for pushing this.
>> > >
>> > > As you mentioned, we have already taken most of the changes into
>> > > production.
>> > > I want to highlight that the main contribution is the addition of
>> > security.
>> > > We have been able to test this at a smaller scale (~500 servers and 4
>> > > subclusters) and the performance is great with our current ZooKeeper
>> > > deployment.
>> > > I would also like to highlight that all the changes are constrained to
>> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
>> > >
>> > > +1 on merging
>> > >
>> > > Inigo
>> > >
>> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks Brahma for starting the discussion.
>> > > > I'm +1 for merging this.
>> > > >
>> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
>> > > > with 2 routers (not in production) and running several tests.
>> > > >
>> > > > Regards,
>> > > > Akira
>> > > >
>> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
>> > brahma@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > Dear Hadoop Developers
>> > > > >
>> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
>> We
>> > > have
>> > > > > been working on this feature from last several months.
>> > > > > This feature work received the contributions from different
>> > companies.
>> > > > All
>> > > > > of the feature development happened smoothly and collaboratively
>> in
>> > > > JIRAs.
>> > > > >
>> > > > > Kindly do take a look at the branch and raise issues/concerns that
>> > need
>> > > > to
>> > > > > be addressed before the merge.
>> > > > >
>> > > > > *Highlights of HDFS-13891 Branch:*
>> > > > > =============================
>> > > > >
>> > > > > Adding Security to RBF(1)
>> > > > > Adding Missing Client API's(2)
>> > > > > Improvements/Bug Fixing
>> > > > > Critical - HDFS-13637, HDFS-13834
>> > > > >
>> > > > > *Commits:*
>> > > > > ========
>> > > > >
>> > > > > No of JIRAs Resolved: 72
>> > > > >
>> > > > > All this commits are in RBF Module. No changes in hdfs/common.
>> > > > >
>> > > > > *Tested Cluster:*
>> > > > > =============
>> > > > >
>> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
>> > other
>> > > > > compaines.
>> > > > >
>> > > > > *Uber*: Most changes are running in production @Uber including the
>> > > > critical
>> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
>> Routers.
>> > > > > Zookeeper as a state store to hold delegation tokens were also
>> stress
>> > > > > tested to hold more than 2 Million tokens. --CR Hota
>> > > > >
>> > > > > *MicroSoft*: Most of these changes are currently running in
>> > production
>> > > at
>> > > > > Microsoft.The security has also been tested in a 500 server
>> cluster
>> > > with
>> > > > 4
>> > > > > subclsuters. --Inigo Goiri
>> > > > >
>> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
>> > > > > routers.Planning deploy 10K production cluster.
>> > > > >
>> > > > > *Contributors:*
>> > > > > ===========
>> > > > >
>> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
>> Asanuma,Shubham
>> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
>> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
>> > > Xiaoqiao,Sherwood
>> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
>> > discussions
>> > > > and
>> > > > > contributing to this.
>> > > > >
>> > > > > *Future Tasks:*
>> > > > > ============
>> > > > >
>> > > > > will cleanup the jira's under this umbrella and contiue to work.
>> > > > >
>> > > > > Reference:
>> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
>> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --Brahma Reddy Battula
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>> Regards,
>> Chitta
>>
>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
Re: [Notice] Merge HDFS-13891(RBF) to trunk
Posted by Brahma Reddy Battula <br...@hotmail.com>.
Merged. thanks to Arpit for quoting the merge process.
________________________________
From: Brahma Reddy Battula <br...@hotmail.com>
Sent: Tuesday, June 25, 2019 12:39 AM
To: hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Cc: Akira Ajisaka; Chittaranjan Hota; Giovanni Matteo Fumarola; Hadoop Common; Hdfs-dev; Iñigo Goiri; Hadoop Common
Subject: [Notice] Merge HDFS-13891(RBF) to trunk
Rebased HDFS-13891 against trunk and ran the jenkins (1).Only two four testcases are failing which flaky and there is already jira to track this. (HDFS-14461)
Going to merge the branch soon which contains 77 commits,so please bear with us.
1.https://builds.apache.org/job/PreCommit-HDFS-Build/27044/console
________________________________
From: Brahma Reddy Battula <br...@apache.org>
Sent: Sunday, June 9, 2019 9:56 PM
To: Xiaoqiao He
Cc: Akira Ajisaka; Chittaranjan Hota; Giovanni Matteo Fumarola; Hadoop Common; Hdfs-dev; Iñigo Goiri
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Hi All,
Given the positive response to the discussion thread [1], here is the
formal vote thread to merge HDFS-13891 in to trunk.
Summary of code changes:
1. Code changes for this branch are done in the hadoop-hdfs-rbf
subproject, there is no impact to hadoop-hdfs and hadoop-common.
2. Added Security support for RBF
3. Added Missing Client Protocol API's
4. Bug Fixes/ Improvments
The vote will run for 7 days, ending Sat June 15th. I will start this
vote with my +1.
Regards,
Brahma Reddy Battula
1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> Thanks Brahma for starting the thread.
> +1 for merging.
>
> He Xiaoqiao
>
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> wrote:
>
>> Thanks Brahma initiating this.
>> +1(non-binding) for merge.
>>
>> @Uber we have almost all changes specially rbf security in production for
>> a
>> while now without issues.
>>
>> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
>> giovanni.fumarola@gmail.com> wrote:
>>
>> > +1 on merging.
>> >
>> > Thanks Brahma for starting the thread.
>> >
>> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
>> >
>> > > Thank you Brahma for pushing this.
>> > >
>> > > As you mentioned, we have already taken most of the changes into
>> > > production.
>> > > I want to highlight that the main contribution is the addition of
>> > security.
>> > > We have been able to test this at a smaller scale (~500 servers and 4
>> > > subclusters) and the performance is great with our current ZooKeeper
>> > > deployment.
>> > > I would also like to highlight that all the changes are constrained to
>> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
>> > >
>> > > +1 on merging
>> > >
>> > > Inigo
>> > >
>> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks Brahma for starting the discussion.
>> > > > I'm +1 for merging this.
>> > > >
>> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
>> > > > with 2 routers (not in production) and running several tests.
>> > > >
>> > > > Regards,
>> > > > Akira
>> > > >
>> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
>> > brahma@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > Dear Hadoop Developers
>> > > > >
>> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
>> We
>> > > have
>> > > > > been working on this feature from last several months.
>> > > > > This feature work received the contributions from different
>> > companies.
>> > > > All
>> > > > > of the feature development happened smoothly and collaboratively
>> in
>> > > > JIRAs.
>> > > > >
>> > > > > Kindly do take a look at the branch and raise issues/concerns that
>> > need
>> > > > to
>> > > > > be addressed before the merge.
>> > > > >
>> > > > > *Highlights of HDFS-13891 Branch:*
>> > > > > =============================
>> > > > >
>> > > > > Adding Security to RBF(1)
>> > > > > Adding Missing Client API's(2)
>> > > > > Improvements/Bug Fixing
>> > > > > Critical - HDFS-13637, HDFS-13834
>> > > > >
>> > > > > *Commits:*
>> > > > > ========
>> > > > >
>> > > > > No of JIRAs Resolved: 72
>> > > > >
>> > > > > All this commits are in RBF Module. No changes in hdfs/common.
>> > > > >
>> > > > > *Tested Cluster:*
>> > > > > =============
>> > > > >
>> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
>> > other
>> > > > > compaines.
>> > > > >
>> > > > > *Uber*: Most changes are running in production @Uber including the
>> > > > critical
>> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
>> Routers.
>> > > > > Zookeeper as a state store to hold delegation tokens were also
>> stress
>> > > > > tested to hold more than 2 Million tokens. --CR Hota
>> > > > >
>> > > > > *MicroSoft*: Most of these changes are currently running in
>> > production
>> > > at
>> > > > > Microsoft.The security has also been tested in a 500 server
>> cluster
>> > > with
>> > > > 4
>> > > > > subclsuters. --Inigo Goiri
>> > > > >
>> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
>> > > > > routers.Planning deploy 10K production cluster.
>> > > > >
>> > > > > *Contributors:*
>> > > > > ===========
>> > > > >
>> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
>> Asanuma,Shubham
>> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
>> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
>> > > Xiaoqiao,Sherwood
>> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
>> > discussions
>> > > > and
>> > > > > contributing to this.
>> > > > >
>> > > > > *Future Tasks:*
>> > > > > ============
>> > > > >
>> > > > > will cleanup the jira's under this umbrella and contiue to work.
>> > > > >
>> > > > > Reference:
>> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
>> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --Brahma Reddy Battula
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>> Regards,
>> Chitta
>>
>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
[Notice] Merge HDFS-13891(RBF) to trunk
Posted by Brahma Reddy Battula <br...@hotmail.com>.
Rebased HDFS-13891 against trunk and ran the jenkins (1).Only two four testcases are failing which flaky and there is already jira to track this. (HDFS-14461)
Going to merge the branch soon which contains 77 commits,so please bear with us.
1.https://builds.apache.org/job/PreCommit-HDFS-Build/27044/console
________________________________
From: Brahma Reddy Battula <br...@apache.org>
Sent: Sunday, June 9, 2019 9:56 PM
To: Xiaoqiao He
Cc: Akira Ajisaka; Chittaranjan Hota; Giovanni Matteo Fumarola; Hadoop Common; Hdfs-dev; Iñigo Goiri
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Hi All,
Given the positive response to the discussion thread [1], here is the
formal vote thread to merge HDFS-13891 in to trunk.
Summary of code changes:
1. Code changes for this branch are done in the hadoop-hdfs-rbf
subproject, there is no impact to hadoop-hdfs and hadoop-common.
2. Added Security support for RBF
3. Added Missing Client Protocol API's
4. Bug Fixes/ Improvments
The vote will run for 7 days, ending Sat June 15th. I will start this
vote with my +1.
Regards,
Brahma Reddy Battula
1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> Thanks Brahma for starting the thread.
> +1 for merging.
>
> He Xiaoqiao
>
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> wrote:
>
>> Thanks Brahma initiating this.
>> +1(non-binding) for merge.
>>
>> @Uber we have almost all changes specially rbf security in production for
>> a
>> while now without issues.
>>
>> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
>> giovanni.fumarola@gmail.com> wrote:
>>
>> > +1 on merging.
>> >
>> > Thanks Brahma for starting the thread.
>> >
>> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
>> >
>> > > Thank you Brahma for pushing this.
>> > >
>> > > As you mentioned, we have already taken most of the changes into
>> > > production.
>> > > I want to highlight that the main contribution is the addition of
>> > security.
>> > > We have been able to test this at a smaller scale (~500 servers and 4
>> > > subclusters) and the performance is great with our current ZooKeeper
>> > > deployment.
>> > > I would also like to highlight that all the changes are constrained to
>> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
>> > >
>> > > +1 on merging
>> > >
>> > > Inigo
>> > >
>> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks Brahma for starting the discussion.
>> > > > I'm +1 for merging this.
>> > > >
>> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
>> > > > with 2 routers (not in production) and running several tests.
>> > > >
>> > > > Regards,
>> > > > Akira
>> > > >
>> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
>> > brahma@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > Dear Hadoop Developers
>> > > > >
>> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
>> We
>> > > have
>> > > > > been working on this feature from last several months.
>> > > > > This feature work received the contributions from different
>> > companies.
>> > > > All
>> > > > > of the feature development happened smoothly and collaboratively
>> in
>> > > > JIRAs.
>> > > > >
>> > > > > Kindly do take a look at the branch and raise issues/concerns that
>> > need
>> > > > to
>> > > > > be addressed before the merge.
>> > > > >
>> > > > > *Highlights of HDFS-13891 Branch:*
>> > > > > =============================
>> > > > >
>> > > > > Adding Security to RBF(1)
>> > > > > Adding Missing Client API's(2)
>> > > > > Improvements/Bug Fixing
>> > > > > Critical - HDFS-13637, HDFS-13834
>> > > > >
>> > > > > *Commits:*
>> > > > > ========
>> > > > >
>> > > > > No of JIRAs Resolved: 72
>> > > > >
>> > > > > All this commits are in RBF Module. No changes in hdfs/common.
>> > > > >
>> > > > > *Tested Cluster:*
>> > > > > =============
>> > > > >
>> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
>> > other
>> > > > > compaines.
>> > > > >
>> > > > > *Uber*: Most changes are running in production @Uber including the
>> > > > critical
>> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
>> Routers.
>> > > > > Zookeeper as a state store to hold delegation tokens were also
>> stress
>> > > > > tested to hold more than 2 Million tokens. --CR Hota
>> > > > >
>> > > > > *MicroSoft*: Most of these changes are currently running in
>> > production
>> > > at
>> > > > > Microsoft.The security has also been tested in a 500 server
>> cluster
>> > > with
>> > > > 4
>> > > > > subclsuters. --Inigo Goiri
>> > > > >
>> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
>> > > > > routers.Planning deploy 10K production cluster.
>> > > > >
>> > > > > *Contributors:*
>> > > > > ===========
>> > > > >
>> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
>> Asanuma,Shubham
>> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
>> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
>> > > Xiaoqiao,Sherwood
>> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
>> > discussions
>> > > > and
>> > > > > contributing to this.
>> > > > >
>> > > > > *Future Tasks:*
>> > > > > ============
>> > > > >
>> > > > > will cleanup the jira's under this umbrella and contiue to work.
>> > > > >
>> > > > > Reference:
>> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
>> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --Brahma Reddy Battula
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>> Regards,
>> Chitta
>>
>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Posted by CR Hota <ch...@gmail.com>.
+1 (non-binding)
On Sun, Jun 9, 2019 at 7:55 PM Akira Ajisaka <aa...@apache.org> wrote:
> +1
>
> -Akira
>
> On Sun, Jun 9, 2019 at 10:56 PM Brahma Reddy Battula <br...@apache.org>
> wrote:
> >
> > Hi All,
> >
> > Given the positive response to the discussion thread [1], here is the
> > formal vote thread to merge HDFS-13891 in to trunk.
> >
> > Summary of code changes:
> > 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> > subproject, there is no impact to hadoop-hdfs and hadoop-common.
> > 2. Added Security support for RBF
> > 3. Added Missing Client Protocol API's
> > 4. Bug Fixes/ Improvments
> >
> >
> > The vote will run for 7 days, ending Sat June 15th. I will start
> this
> > vote with my +1.
> >
> > Regards,
> > Brahma Reddy Battula
> >
> > 1).
> >
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
> >
> >
> > On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> >
> > > Thanks Brahma for starting the thread.
> > > +1 for merging.
> > >
> > > He Xiaoqiao
> > >
> > > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <
> chitts.hota@gmail.com>
> > > wrote:
> > >
> > >> Thanks Brahma initiating this.
> > >> +1(non-binding) for merge.
> > >>
> > >> @Uber we have almost all changes specially rbf security in production
> for
> > >> a
> > >> while now without issues.
> > >>
> > >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> > >> giovanni.fumarola@gmail.com> wrote:
> > >>
> > >> > +1 on merging.
> > >> >
> > >> > Thanks Brahma for starting the thread.
> > >> >
> > >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com>
> wrote:
> > >> >
> > >> > > Thank you Brahma for pushing this.
> > >> > >
> > >> > > As you mentioned, we have already taken most of the changes into
> > >> > > production.
> > >> > > I want to highlight that the main contribution is the addition of
> > >> > security.
> > >> > > We have been able to test this at a smaller scale (~500 servers
> and 4
> > >> > > subclusters) and the performance is great with our current
> ZooKeeper
> > >> > > deployment.
> > >> > > I would also like to highlight that all the changes are
> constrained to
> > >> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
> > >> > >
> > >> > > +1 on merging
> > >> > >
> > >> > > Inigo
> > >> > >
> > >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aajisaka@gmail.com
> >
> > >> > wrote:
> > >> > >
> > >> > > > Thanks Brahma for starting the discussion.
> > >> > > > I'm +1 for merging this.
> > >> > > >
> > >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
> cluster
> > >> > > > with 2 routers (not in production) and running several tests.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Akira
> > >> > > >
> > >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> > >> > brahma@apache.org>
> > >> > > > wrote:
> > >> > > > >
> > >> > > > > Dear Hadoop Developers
> > >> > > > >
> > >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into
> trunk.
> > >> We
> > >> > > have
> > >> > > > > been working on this feature from last several months.
> > >> > > > > This feature work received the contributions from different
> > >> > companies.
> > >> > > > All
> > >> > > > > of the feature development happened smoothly and
> collaboratively
> > >> in
> > >> > > > JIRAs.
> > >> > > > >
> > >> > > > > Kindly do take a look at the branch and raise issues/concerns
> that
> > >> > need
> > >> > > > to
> > >> > > > > be addressed before the merge.
> > >> > > > >
> > >> > > > > *Highlights of HDFS-13891 Branch:*
> > >> > > > > =============================
> > >> > > > >
> > >> > > > > Adding Security to RBF(1)
> > >> > > > > Adding Missing Client API's(2)
> > >> > > > > Improvements/Bug Fixing
> > >> > > > > Critical - HDFS-13637, HDFS-13834
> > >> > > > >
> > >> > > > > *Commits:*
> > >> > > > > ========
> > >> > > > >
> > >> > > > > No of JIRAs Resolved: 72
> > >> > > > >
> > >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> > >> > > > >
> > >> > > > > *Tested Cluster:*
> > >> > > > > =============
> > >> > > > >
> > >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and
> some
> > >> > other
> > >> > > > > compaines.
> > >> > > > >
> > >> > > > > *Uber*: Most changes are running in production @Uber
> including the
> > >> > > > critical
> > >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> > >> Routers.
> > >> > > > > Zookeeper as a state store to hold delegation tokens were also
> > >> stress
> > >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> > >> > > > >
> > >> > > > > *MicroSoft*: Most of these changes are currently running in
> > >> > production
> > >> > > at
> > >> > > > > Microsoft.The security has also been tested in a 500 server
> > >> cluster
> > >> > > with
> > >> > > > 4
> > >> > > > > subclsuters. --Inigo Goiri
> > >> > > > >
> > >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
> > >> > > > > routers.Planning deploy 10K production cluster.
> > >> > > > >
> > >> > > > > *Contributors:*
> > >> > > > > ===========
> > >> > > > >
> > >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> > >> Asanuma,Shubham
> > >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
> Li,Gang
> > >> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
> > >> > > Xiaoqiao,Sherwood
> > >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> > >> > discussions
> > >> > > > and
> > >> > > > > contributing to this.
> > >> > > > >
> > >> > > > > *Future Tasks:*
> > >> > > > > ============
> > >> > > > >
> > >> > > > > will cleanup the jira's under this umbrella and contiue to
> work.
> > >> > > > >
> > >> > > > > Reference:
> > >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> > >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --Brahma Reddy Battula
> > >> > > >
> > >> > > >
> > >> ---------------------------------------------------------------------
> > >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > >> > > > For additional commands, e-mail:
> hdfs-dev-help@hadoop.apache.org
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >> --
> > >> Regards,
> > >> Chitta
> > >>
> > >
> >
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> > --
> >
> >
> >
> > --Brahma Reddy Battula
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
--
Regards,
Chitta
Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Akira Ajisaka <aa...@apache.org>.
+1
-Akira
On Sun, Jun 9, 2019 at 10:56 PM Brahma Reddy Battula <br...@apache.org> wrote:
>
> Hi All,
>
> Given the positive response to the discussion thread [1], here is the
> formal vote thread to merge HDFS-13891 in to trunk.
>
> Summary of code changes:
> 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> subproject, there is no impact to hadoop-hdfs and hadoop-common.
> 2. Added Security support for RBF
> 3. Added Missing Client Protocol API's
> 4. Bug Fixes/ Improvments
>
>
> The vote will run for 7 days, ending Sat June 15th. I will start this
> vote with my +1.
>
> Regards,
> Brahma Reddy Battula
>
> 1).
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
>
>
> On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
>
> > Thanks Brahma for starting the thread.
> > +1 for merging.
> >
> > He Xiaoqiao
> >
> > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> > wrote:
> >
> >> Thanks Brahma initiating this.
> >> +1(non-binding) for merge.
> >>
> >> @Uber we have almost all changes specially rbf security in production for
> >> a
> >> while now without issues.
> >>
> >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> >> giovanni.fumarola@gmail.com> wrote:
> >>
> >> > +1 on merging.
> >> >
> >> > Thanks Brahma for starting the thread.
> >> >
> >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
> >> >
> >> > > Thank you Brahma for pushing this.
> >> > >
> >> > > As you mentioned, we have already taken most of the changes into
> >> > > production.
> >> > > I want to highlight that the main contribution is the addition of
> >> > security.
> >> > > We have been able to test this at a smaller scale (~500 servers and 4
> >> > > subclusters) and the performance is great with our current ZooKeeper
> >> > > deployment.
> >> > > I would also like to highlight that all the changes are constrained to
> >> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
> >> > >
> >> > > +1 on merging
> >> > >
> >> > > Inigo
> >> > >
> >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Thanks Brahma for starting the discussion.
> >> > > > I'm +1 for merging this.
> >> > > >
> >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
> >> > > > with 2 routers (not in production) and running several tests.
> >> > > >
> >> > > > Regards,
> >> > > > Akira
> >> > > >
> >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> >> > brahma@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > Dear Hadoop Developers
> >> > > > >
> >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
> >> We
> >> > > have
> >> > > > > been working on this feature from last several months.
> >> > > > > This feature work received the contributions from different
> >> > companies.
> >> > > > All
> >> > > > > of the feature development happened smoothly and collaboratively
> >> in
> >> > > > JIRAs.
> >> > > > >
> >> > > > > Kindly do take a look at the branch and raise issues/concerns that
> >> > need
> >> > > > to
> >> > > > > be addressed before the merge.
> >> > > > >
> >> > > > > *Highlights of HDFS-13891 Branch:*
> >> > > > > =============================
> >> > > > >
> >> > > > > Adding Security to RBF(1)
> >> > > > > Adding Missing Client API's(2)
> >> > > > > Improvements/Bug Fixing
> >> > > > > Critical - HDFS-13637, HDFS-13834
> >> > > > >
> >> > > > > *Commits:*
> >> > > > > ========
> >> > > > >
> >> > > > > No of JIRAs Resolved: 72
> >> > > > >
> >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> >> > > > >
> >> > > > > *Tested Cluster:*
> >> > > > > =============
> >> > > > >
> >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
> >> > other
> >> > > > > compaines.
> >> > > > >
> >> > > > > *Uber*: Most changes are running in production @Uber including the
> >> > > > critical
> >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> >> Routers.
> >> > > > > Zookeeper as a state store to hold delegation tokens were also
> >> stress
> >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> >> > > > >
> >> > > > > *MicroSoft*: Most of these changes are currently running in
> >> > production
> >> > > at
> >> > > > > Microsoft.The security has also been tested in a 500 server
> >> cluster
> >> > > with
> >> > > > 4
> >> > > > > subclsuters. --Inigo Goiri
> >> > > > >
> >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
> >> > > > > routers.Planning deploy 10K production cluster.
> >> > > > >
> >> > > > > *Contributors:*
> >> > > > > ===========
> >> > > > >
> >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> >> Asanuma,Shubham
> >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
> >> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
> >> > > Xiaoqiao,Sherwood
> >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> >> > discussions
> >> > > > and
> >> > > > > contributing to this.
> >> > > > >
> >> > > > > *Future Tasks:*
> >> > > > > ============
> >> > > > >
> >> > > > > will cleanup the jira's under this umbrella and contiue to work.
> >> > > > >
> >> > > > > Reference:
> >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --Brahma Reddy Battula
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >> Regards,
> >> Chitta
> >>
> >
>
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
[RESULT][VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Brahma Reddy Battula <br...@apache.org>.
Vote is passed with 3 binding +1's and no -1 ( 5 nonbinding votes) .
thanks to all for voting.
Will try to merge this week..
On Tue, Jun 11, 2019 at 1:13 PM Takanobu Asanuma <ta...@yahoo-corp.jp>
wrote:
> +1(non-binding).
>
> Regards,
> - Takanobu
>
> ________________________________________
> From: Xiaoqiao He <xq...@gmail.com>
> Sent: Monday, June 10, 2019 15:47
> To: Ranith Sardar
> Cc: Brahma Reddy Battula; Hadoop Common; Hdfs-dev
> Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
>
> +1 (non-binding)
>
> - Try to merge branch HDFS-13891(RBF) to trunk at local and no conflict or
> failure.
> - Built from merged sources.
> - Ran 85 RBF test class at local and result shows: (Tests run: 639,
> Failures: 2, Errors: 2, Skipped: 2) - failed tests include
> #TestRouterWithSecureStartup and #TestRouterHttpDelegationToken. I don't
> think it is blocked issue.
>
> Thanks Brahma for organizing the merge.
>
>
> On Mon, Jun 10, 2019 at 12:27 PM Ranith Sardar <ra...@huawei.com>
> wrote:
>
> > +1 (Non-binding)
> >
> > -----Original Message-----
> > From: Brahma Reddy Battula [mailto:brahma@apache.org]
> > Sent: 09 June 2019 19:31
> > To: Hadoop Common <co...@hadoop.apache.org>; Hdfs-dev <
> > hdfs-dev@hadoop.apache.org>
> > Subject: [VOTE] Merge HDFS-13891(RBF) to trunk
> >
> > Updated mail...
> >
> > ---------- Forwarded message ---------
> > From: Brahma Reddy Battula <br...@apache.org>
> > Date: Sun, Jun 9, 2019 at 7:26 PM
> > Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
> > To: Xiaoqiao He <xq...@gmail.com>
> > Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota <
> > chitts.hota@gmail.com>, Giovanni Matteo Fumarola <
> > giovanni.fumarola@gmail.com>, Hadoop Common <
> common-dev@hadoop.apache.org>,
> > Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
> >
> >
> > Hi All,
> >
> > Given the positive response to the discussion thread [1], here is the
> > formal vote thread to merge HDFS-13891 in to trunk.
> >
> > Summary of code changes:
> > 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> > subproject, there is no impact to hadoop-hdfs and hadoop-common.
> > 2. Added Security support for RBF
> > 3. Added Missing Client Protocol API's
> > 4. Bug Fixes/ Improvments
> >
> >
> > The vote will run for 7 days, ending Sat June 15th. I will start
> this
> > vote with my +1.
> >
> > Regards,
> > Brahma Reddy Battula
> >
> > 1).
> >
> >
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
> >
> >
> > On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> >
> > > Thanks Brahma for starting the thread.
> > > +1 for merging.
> > >
> > > He Xiaoqiao
> > >
> > > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota
> > > <ch...@gmail.com>
> > > wrote:
> > >
> > >> Thanks Brahma initiating this.
> > >> +1(non-binding) for merge.
> > >>
> > >> @Uber we have almost all changes specially rbf security in production
> > >> for a while now without issues.
> > >>
> > >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> > >> giovanni.fumarola@gmail.com> wrote:
> > >>
> > >> > +1 on merging.
> > >> >
> > >> > Thanks Brahma for starting the thread.
> > >> >
> > >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com>
> > wrote:
> > >> >
> > >> > > Thank you Brahma for pushing this.
> > >> > >
> > >> > > As you mentioned, we have already taken most of the changes into
> > >> > > production.
> > >> > > I want to highlight that the main contribution is the addition of
> > >> > security.
> > >> > > We have been able to test this at a smaller scale (~500 servers
> > >> > > and 4
> > >> > > subclusters) and the performance is great with our current
> > >> > > ZooKeeper deployment.
> > >> > > I would also like to highlight that all the changes are
> > >> > > constrained to hadoop-hdfs-rbf and there is no differences in
> > commons or HDFS.
> > >> > >
> > >> > > +1 on merging
> > >> > >
> > >> > > Inigo
> > >> > >
> > >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka
> > >> > > <aa...@gmail.com>
> > >> > wrote:
> > >> > >
> > >> > > > Thanks Brahma for starting the discussion.
> > >> > > > I'm +1 for merging this.
> > >> > > >
> > >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
> > >> > > > cluster with 2 routers (not in production) and running several
> > tests.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Akira
> > >> > > >
> > >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> > >> > brahma@apache.org>
> > >> > > > wrote:
> > >> > > > >
> > >> > > > > Dear Hadoop Developers
> > >> > > > >
> > >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into
> > trunk.
> > >> We
> > >> > > have
> > >> > > > > been working on this feature from last several months.
> > >> > > > > This feature work received the contributions from different
> > >> > companies.
> > >> > > > All
> > >> > > > > of the feature development happened smoothly and
> > >> > > > > collaboratively
> > >> in
> > >> > > > JIRAs.
> > >> > > > >
> > >> > > > > Kindly do take a look at the branch and raise issues/concerns
> > >> > > > > that
> > >> > need
> > >> > > > to
> > >> > > > > be addressed before the merge.
> > >> > > > >
> > >> > > > > *Highlights of HDFS-13891 Branch:*
> > >> > > > > =============================
> > >> > > > >
> > >> > > > > Adding Security to RBF(1)
> > >> > > > > Adding Missing Client API's(2) Improvements/Bug Fixing
> > >> > > > > Critical - HDFS-13637, HDFS-13834
> > >> > > > >
> > >> > > > > *Commits:*
> > >> > > > > ========
> > >> > > > >
> > >> > > > > No of JIRAs Resolved: 72
> > >> > > > >
> > >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> > >> > > > >
> > >> > > > > *Tested Cluster:*
> > >> > > > > =============
> > >> > > > >
> > >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and
> > >> > > > > some
> > >> > other
> > >> > > > > compaines.
> > >> > > > >
> > >> > > > > *Uber*: Most changes are running in production @Uber
> > >> > > > > including the
> > >> > > > critical
> > >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> > >> Routers.
> > >> > > > > Zookeeper as a state store to hold delegation tokens were
> > >> > > > > also
> > >> stress
> > >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> > >> > > > >
> > >> > > > > *MicroSoft*: Most of these changes are currently running in
> > >> > production
> > >> > > at
> > >> > > > > Microsoft.The security has also been tested in a 500 server
> > >> cluster
> > >> > > with
> > >> > > > 4
> > >> > > > > subclsuters. --Inigo Goiri
> > >> > > > >
> > >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with
> > >> > > > > 3 routers.Planning deploy 10K production cluster.
> > >> > > > >
> > >> > > > > *Contributors:*
> > >> > > > > ===========
> > >> > > > >
> > >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> > >> Asanuma,Shubham
> > >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
> > >> > > > > Li,Gang Li,Surendra Singh Lihore,Ranith Sardar,Ayush
> > >> > > > > Saxena,He
> > >> > > Xiaoqiao,Sherwood
> > >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> > >> > discussions
> > >> > > > and
> > >> > > > > contributing to this.
> > >> > > > >
> > >> > > > > *Future Tasks:*
> > >> > > > > ============
> > >> > > > >
> > >> > > > > will cleanup the jira's under this umbrella and contiue to
> > work.
> > >> > > > >
> > >> > > > > Reference:
> > >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> > >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --Brahma Reddy Battula
> > >> > > >
> > >> > > >
> > >> ---------------------------------------------------------------------
> > >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > >> > > > For additional commands, e-mail:
> > >> > > > hdfs-dev-help@hadoop.apache.org
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >> --
> > >> Regards,
> > >> Chitta
> > >
> > >
> > >>
> >
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> >
>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
[RESULT][VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Brahma Reddy Battula <br...@apache.org>.
Vote is passed with 3 binding +1's and no -1 ( 5 nonbinding votes) .
thanks to all for voting.
Will try to merge this week..
On Tue, Jun 11, 2019 at 1:13 PM Takanobu Asanuma <ta...@yahoo-corp.jp>
wrote:
> +1(non-binding).
>
> Regards,
> - Takanobu
>
> ________________________________________
> From: Xiaoqiao He <xq...@gmail.com>
> Sent: Monday, June 10, 2019 15:47
> To: Ranith Sardar
> Cc: Brahma Reddy Battula; Hadoop Common; Hdfs-dev
> Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
>
> +1 (non-binding)
>
> - Try to merge branch HDFS-13891(RBF) to trunk at local and no conflict or
> failure.
> - Built from merged sources.
> - Ran 85 RBF test class at local and result shows: (Tests run: 639,
> Failures: 2, Errors: 2, Skipped: 2) - failed tests include
> #TestRouterWithSecureStartup and #TestRouterHttpDelegationToken. I don't
> think it is blocked issue.
>
> Thanks Brahma for organizing the merge.
>
>
> On Mon, Jun 10, 2019 at 12:27 PM Ranith Sardar <ra...@huawei.com>
> wrote:
>
> > +1 (Non-binding)
> >
> > -----Original Message-----
> > From: Brahma Reddy Battula [mailto:brahma@apache.org]
> > Sent: 09 June 2019 19:31
> > To: Hadoop Common <co...@hadoop.apache.org>; Hdfs-dev <
> > hdfs-dev@hadoop.apache.org>
> > Subject: [VOTE] Merge HDFS-13891(RBF) to trunk
> >
> > Updated mail...
> >
> > ---------- Forwarded message ---------
> > From: Brahma Reddy Battula <br...@apache.org>
> > Date: Sun, Jun 9, 2019 at 7:26 PM
> > Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
> > To: Xiaoqiao He <xq...@gmail.com>
> > Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota <
> > chitts.hota@gmail.com>, Giovanni Matteo Fumarola <
> > giovanni.fumarola@gmail.com>, Hadoop Common <
> common-dev@hadoop.apache.org>,
> > Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
> >
> >
> > Hi All,
> >
> > Given the positive response to the discussion thread [1], here is the
> > formal vote thread to merge HDFS-13891 in to trunk.
> >
> > Summary of code changes:
> > 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> > subproject, there is no impact to hadoop-hdfs and hadoop-common.
> > 2. Added Security support for RBF
> > 3. Added Missing Client Protocol API's
> > 4. Bug Fixes/ Improvments
> >
> >
> > The vote will run for 7 days, ending Sat June 15th. I will start
> this
> > vote with my +1.
> >
> > Regards,
> > Brahma Reddy Battula
> >
> > 1).
> >
> >
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
> >
> >
> > On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> >
> > > Thanks Brahma for starting the thread.
> > > +1 for merging.
> > >
> > > He Xiaoqiao
> > >
> > > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota
> > > <ch...@gmail.com>
> > > wrote:
> > >
> > >> Thanks Brahma initiating this.
> > >> +1(non-binding) for merge.
> > >>
> > >> @Uber we have almost all changes specially rbf security in production
> > >> for a while now without issues.
> > >>
> > >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> > >> giovanni.fumarola@gmail.com> wrote:
> > >>
> > >> > +1 on merging.
> > >> >
> > >> > Thanks Brahma for starting the thread.
> > >> >
> > >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com>
> > wrote:
> > >> >
> > >> > > Thank you Brahma for pushing this.
> > >> > >
> > >> > > As you mentioned, we have already taken most of the changes into
> > >> > > production.
> > >> > > I want to highlight that the main contribution is the addition of
> > >> > security.
> > >> > > We have been able to test this at a smaller scale (~500 servers
> > >> > > and 4
> > >> > > subclusters) and the performance is great with our current
> > >> > > ZooKeeper deployment.
> > >> > > I would also like to highlight that all the changes are
> > >> > > constrained to hadoop-hdfs-rbf and there is no differences in
> > commons or HDFS.
> > >> > >
> > >> > > +1 on merging
> > >> > >
> > >> > > Inigo
> > >> > >
> > >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka
> > >> > > <aa...@gmail.com>
> > >> > wrote:
> > >> > >
> > >> > > > Thanks Brahma for starting the discussion.
> > >> > > > I'm +1 for merging this.
> > >> > > >
> > >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
> > >> > > > cluster with 2 routers (not in production) and running several
> > tests.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Akira
> > >> > > >
> > >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> > >> > brahma@apache.org>
> > >> > > > wrote:
> > >> > > > >
> > >> > > > > Dear Hadoop Developers
> > >> > > > >
> > >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into
> > trunk.
> > >> We
> > >> > > have
> > >> > > > > been working on this feature from last several months.
> > >> > > > > This feature work received the contributions from different
> > >> > companies.
> > >> > > > All
> > >> > > > > of the feature development happened smoothly and
> > >> > > > > collaboratively
> > >> in
> > >> > > > JIRAs.
> > >> > > > >
> > >> > > > > Kindly do take a look at the branch and raise issues/concerns
> > >> > > > > that
> > >> > need
> > >> > > > to
> > >> > > > > be addressed before the merge.
> > >> > > > >
> > >> > > > > *Highlights of HDFS-13891 Branch:*
> > >> > > > > =============================
> > >> > > > >
> > >> > > > > Adding Security to RBF(1)
> > >> > > > > Adding Missing Client API's(2) Improvements/Bug Fixing
> > >> > > > > Critical - HDFS-13637, HDFS-13834
> > >> > > > >
> > >> > > > > *Commits:*
> > >> > > > > ========
> > >> > > > >
> > >> > > > > No of JIRAs Resolved: 72
> > >> > > > >
> > >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> > >> > > > >
> > >> > > > > *Tested Cluster:*
> > >> > > > > =============
> > >> > > > >
> > >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and
> > >> > > > > some
> > >> > other
> > >> > > > > compaines.
> > >> > > > >
> > >> > > > > *Uber*: Most changes are running in production @Uber
> > >> > > > > including the
> > >> > > > critical
> > >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> > >> Routers.
> > >> > > > > Zookeeper as a state store to hold delegation tokens were
> > >> > > > > also
> > >> stress
> > >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> > >> > > > >
> > >> > > > > *MicroSoft*: Most of these changes are currently running in
> > >> > production
> > >> > > at
> > >> > > > > Microsoft.The security has also been tested in a 500 server
> > >> cluster
> > >> > > with
> > >> > > > 4
> > >> > > > > subclsuters. --Inigo Goiri
> > >> > > > >
> > >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with
> > >> > > > > 3 routers.Planning deploy 10K production cluster.
> > >> > > > >
> > >> > > > > *Contributors:*
> > >> > > > > ===========
> > >> > > > >
> > >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> > >> Asanuma,Shubham
> > >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
> > >> > > > > Li,Gang Li,Surendra Singh Lihore,Ranith Sardar,Ayush
> > >> > > > > Saxena,He
> > >> > > Xiaoqiao,Sherwood
> > >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> > >> > discussions
> > >> > > > and
> > >> > > > > contributing to this.
> > >> > > > >
> > >> > > > > *Future Tasks:*
> > >> > > > > ============
> > >> > > > >
> > >> > > > > will cleanup the jira's under this umbrella and contiue to
> > work.
> > >> > > > >
> > >> > > > > Reference:
> > >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> > >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --Brahma Reddy Battula
> > >> > > >
> > >> > > >
> > >> ---------------------------------------------------------------------
> > >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > >> > > > For additional commands, e-mail:
> > >> > > > hdfs-dev-help@hadoop.apache.org
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >> --
> > >> Regards,
> > >> Chitta
> > >
> > >
> > >>
> >
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> >
>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Takanobu Asanuma <ta...@yahoo-corp.jp>.
+1(non-binding).
Regards,
- Takanobu
________________________________________
From: Xiaoqiao He <xq...@gmail.com>
Sent: Monday, June 10, 2019 15:47
To: Ranith Sardar
Cc: Brahma Reddy Battula; Hadoop Common; Hdfs-dev
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
+1 (non-binding)
- Try to merge branch HDFS-13891(RBF) to trunk at local and no conflict or
failure.
- Built from merged sources.
- Ran 85 RBF test class at local and result shows: (Tests run: 639,
Failures: 2, Errors: 2, Skipped: 2) - failed tests include
#TestRouterWithSecureStartup and #TestRouterHttpDelegationToken. I don't
think it is blocked issue.
Thanks Brahma for organizing the merge.
On Mon, Jun 10, 2019 at 12:27 PM Ranith Sardar <ra...@huawei.com>
wrote:
> +1 (Non-binding)
>
> -----Original Message-----
> From: Brahma Reddy Battula [mailto:brahma@apache.org]
> Sent: 09 June 2019 19:31
> To: Hadoop Common <co...@hadoop.apache.org>; Hdfs-dev <
> hdfs-dev@hadoop.apache.org>
> Subject: [VOTE] Merge HDFS-13891(RBF) to trunk
>
> Updated mail...
>
> ---------- Forwarded message ---------
> From: Brahma Reddy Battula <br...@apache.org>
> Date: Sun, Jun 9, 2019 at 7:26 PM
> Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
> To: Xiaoqiao He <xq...@gmail.com>
> Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota <
> chitts.hota@gmail.com>, Giovanni Matteo Fumarola <
> giovanni.fumarola@gmail.com>, Hadoop Common <co...@hadoop.apache.org>,
> Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
>
>
> Hi All,
>
> Given the positive response to the discussion thread [1], here is the
> formal vote thread to merge HDFS-13891 in to trunk.
>
> Summary of code changes:
> 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> subproject, there is no impact to hadoop-hdfs and hadoop-common.
> 2. Added Security support for RBF
> 3. Added Missing Client Protocol API's
> 4. Bug Fixes/ Improvments
>
>
> The vote will run for 7 days, ending Sat June 15th. I will start this
> vote with my +1.
>
> Regards,
> Brahma Reddy Battula
>
> 1).
>
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
>
>
> On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
>
> > Thanks Brahma for starting the thread.
> > +1 for merging.
> >
> > He Xiaoqiao
> >
> > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota
> > <ch...@gmail.com>
> > wrote:
> >
> >> Thanks Brahma initiating this.
> >> +1(non-binding) for merge.
> >>
> >> @Uber we have almost all changes specially rbf security in production
> >> for a while now without issues.
> >>
> >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> >> giovanni.fumarola@gmail.com> wrote:
> >>
> >> > +1 on merging.
> >> >
> >> > Thanks Brahma for starting the thread.
> >> >
> >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com>
> wrote:
> >> >
> >> > > Thank you Brahma for pushing this.
> >> > >
> >> > > As you mentioned, we have already taken most of the changes into
> >> > > production.
> >> > > I want to highlight that the main contribution is the addition of
> >> > security.
> >> > > We have been able to test this at a smaller scale (~500 servers
> >> > > and 4
> >> > > subclusters) and the performance is great with our current
> >> > > ZooKeeper deployment.
> >> > > I would also like to highlight that all the changes are
> >> > > constrained to hadoop-hdfs-rbf and there is no differences in
> commons or HDFS.
> >> > >
> >> > > +1 on merging
> >> > >
> >> > > Inigo
> >> > >
> >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka
> >> > > <aa...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Thanks Brahma for starting the discussion.
> >> > > > I'm +1 for merging this.
> >> > > >
> >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
> >> > > > cluster with 2 routers (not in production) and running several
> tests.
> >> > > >
> >> > > > Regards,
> >> > > > Akira
> >> > > >
> >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> >> > brahma@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > Dear Hadoop Developers
> >> > > > >
> >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into
> trunk.
> >> We
> >> > > have
> >> > > > > been working on this feature from last several months.
> >> > > > > This feature work received the contributions from different
> >> > companies.
> >> > > > All
> >> > > > > of the feature development happened smoothly and
> >> > > > > collaboratively
> >> in
> >> > > > JIRAs.
> >> > > > >
> >> > > > > Kindly do take a look at the branch and raise issues/concerns
> >> > > > > that
> >> > need
> >> > > > to
> >> > > > > be addressed before the merge.
> >> > > > >
> >> > > > > *Highlights of HDFS-13891 Branch:*
> >> > > > > =============================
> >> > > > >
> >> > > > > Adding Security to RBF(1)
> >> > > > > Adding Missing Client API's(2) Improvements/Bug Fixing
> >> > > > > Critical - HDFS-13637, HDFS-13834
> >> > > > >
> >> > > > > *Commits:*
> >> > > > > ========
> >> > > > >
> >> > > > > No of JIRAs Resolved: 72
> >> > > > >
> >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> >> > > > >
> >> > > > > *Tested Cluster:*
> >> > > > > =============
> >> > > > >
> >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and
> >> > > > > some
> >> > other
> >> > > > > compaines.
> >> > > > >
> >> > > > > *Uber*: Most changes are running in production @Uber
> >> > > > > including the
> >> > > > critical
> >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> >> Routers.
> >> > > > > Zookeeper as a state store to hold delegation tokens were
> >> > > > > also
> >> stress
> >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> >> > > > >
> >> > > > > *MicroSoft*: Most of these changes are currently running in
> >> > production
> >> > > at
> >> > > > > Microsoft.The security has also been tested in a 500 server
> >> cluster
> >> > > with
> >> > > > 4
> >> > > > > subclsuters. --Inigo Goiri
> >> > > > >
> >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with
> >> > > > > 3 routers.Planning deploy 10K production cluster.
> >> > > > >
> >> > > > > *Contributors:*
> >> > > > > ===========
> >> > > > >
> >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> >> Asanuma,Shubham
> >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
> >> > > > > Li,Gang Li,Surendra Singh Lihore,Ranith Sardar,Ayush
> >> > > > > Saxena,He
> >> > > Xiaoqiao,Sherwood
> >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> >> > discussions
> >> > > > and
> >> > > > > contributing to this.
> >> > > > >
> >> > > > > *Future Tasks:*
> >> > > > > ============
> >> > > > >
> >> > > > > will cleanup the jira's under this umbrella and contiue to
> work.
> >> > > > >
> >> > > > > Reference:
> >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --Brahma Reddy Battula
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > For additional commands, e-mail:
> >> > > > hdfs-dev-help@hadoop.apache.org
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >> Regards,
> >> Chitta
> >
> >
> >>
>
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
>
Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Takanobu Asanuma <ta...@yahoo-corp.jp>.
+1(non-binding).
Regards,
- Takanobu
________________________________________
From: Xiaoqiao He <xq...@gmail.com>
Sent: Monday, June 10, 2019 15:47
To: Ranith Sardar
Cc: Brahma Reddy Battula; Hadoop Common; Hdfs-dev
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
+1 (non-binding)
- Try to merge branch HDFS-13891(RBF) to trunk at local and no conflict or
failure.
- Built from merged sources.
- Ran 85 RBF test class at local and result shows: (Tests run: 639,
Failures: 2, Errors: 2, Skipped: 2) - failed tests include
#TestRouterWithSecureStartup and #TestRouterHttpDelegationToken. I don't
think it is blocked issue.
Thanks Brahma for organizing the merge.
On Mon, Jun 10, 2019 at 12:27 PM Ranith Sardar <ra...@huawei.com>
wrote:
> +1 (Non-binding)
>
> -----Original Message-----
> From: Brahma Reddy Battula [mailto:brahma@apache.org]
> Sent: 09 June 2019 19:31
> To: Hadoop Common <co...@hadoop.apache.org>; Hdfs-dev <
> hdfs-dev@hadoop.apache.org>
> Subject: [VOTE] Merge HDFS-13891(RBF) to trunk
>
> Updated mail...
>
> ---------- Forwarded message ---------
> From: Brahma Reddy Battula <br...@apache.org>
> Date: Sun, Jun 9, 2019 at 7:26 PM
> Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
> To: Xiaoqiao He <xq...@gmail.com>
> Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota <
> chitts.hota@gmail.com>, Giovanni Matteo Fumarola <
> giovanni.fumarola@gmail.com>, Hadoop Common <co...@hadoop.apache.org>,
> Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
>
>
> Hi All,
>
> Given the positive response to the discussion thread [1], here is the
> formal vote thread to merge HDFS-13891 in to trunk.
>
> Summary of code changes:
> 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> subproject, there is no impact to hadoop-hdfs and hadoop-common.
> 2. Added Security support for RBF
> 3. Added Missing Client Protocol API's
> 4. Bug Fixes/ Improvments
>
>
> The vote will run for 7 days, ending Sat June 15th. I will start this
> vote with my +1.
>
> Regards,
> Brahma Reddy Battula
>
> 1).
>
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
>
>
> On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
>
> > Thanks Brahma for starting the thread.
> > +1 for merging.
> >
> > He Xiaoqiao
> >
> > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota
> > <ch...@gmail.com>
> > wrote:
> >
> >> Thanks Brahma initiating this.
> >> +1(non-binding) for merge.
> >>
> >> @Uber we have almost all changes specially rbf security in production
> >> for a while now without issues.
> >>
> >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> >> giovanni.fumarola@gmail.com> wrote:
> >>
> >> > +1 on merging.
> >> >
> >> > Thanks Brahma for starting the thread.
> >> >
> >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com>
> wrote:
> >> >
> >> > > Thank you Brahma for pushing this.
> >> > >
> >> > > As you mentioned, we have already taken most of the changes into
> >> > > production.
> >> > > I want to highlight that the main contribution is the addition of
> >> > security.
> >> > > We have been able to test this at a smaller scale (~500 servers
> >> > > and 4
> >> > > subclusters) and the performance is great with our current
> >> > > ZooKeeper deployment.
> >> > > I would also like to highlight that all the changes are
> >> > > constrained to hadoop-hdfs-rbf and there is no differences in
> commons or HDFS.
> >> > >
> >> > > +1 on merging
> >> > >
> >> > > Inigo
> >> > >
> >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka
> >> > > <aa...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Thanks Brahma for starting the discussion.
> >> > > > I'm +1 for merging this.
> >> > > >
> >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
> >> > > > cluster with 2 routers (not in production) and running several
> tests.
> >> > > >
> >> > > > Regards,
> >> > > > Akira
> >> > > >
> >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> >> > brahma@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > Dear Hadoop Developers
> >> > > > >
> >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into
> trunk.
> >> We
> >> > > have
> >> > > > > been working on this feature from last several months.
> >> > > > > This feature work received the contributions from different
> >> > companies.
> >> > > > All
> >> > > > > of the feature development happened smoothly and
> >> > > > > collaboratively
> >> in
> >> > > > JIRAs.
> >> > > > >
> >> > > > > Kindly do take a look at the branch and raise issues/concerns
> >> > > > > that
> >> > need
> >> > > > to
> >> > > > > be addressed before the merge.
> >> > > > >
> >> > > > > *Highlights of HDFS-13891 Branch:*
> >> > > > > =============================
> >> > > > >
> >> > > > > Adding Security to RBF(1)
> >> > > > > Adding Missing Client API's(2) Improvements/Bug Fixing
> >> > > > > Critical - HDFS-13637, HDFS-13834
> >> > > > >
> >> > > > > *Commits:*
> >> > > > > ========
> >> > > > >
> >> > > > > No of JIRAs Resolved: 72
> >> > > > >
> >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> >> > > > >
> >> > > > > *Tested Cluster:*
> >> > > > > =============
> >> > > > >
> >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and
> >> > > > > some
> >> > other
> >> > > > > compaines.
> >> > > > >
> >> > > > > *Uber*: Most changes are running in production @Uber
> >> > > > > including the
> >> > > > critical
> >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> >> Routers.
> >> > > > > Zookeeper as a state store to hold delegation tokens were
> >> > > > > also
> >> stress
> >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> >> > > > >
> >> > > > > *MicroSoft*: Most of these changes are currently running in
> >> > production
> >> > > at
> >> > > > > Microsoft.The security has also been tested in a 500 server
> >> cluster
> >> > > with
> >> > > > 4
> >> > > > > subclsuters. --Inigo Goiri
> >> > > > >
> >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with
> >> > > > > 3 routers.Planning deploy 10K production cluster.
> >> > > > >
> >> > > > > *Contributors:*
> >> > > > > ===========
> >> > > > >
> >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> >> Asanuma,Shubham
> >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
> >> > > > > Li,Gang Li,Surendra Singh Lihore,Ranith Sardar,Ayush
> >> > > > > Saxena,He
> >> > > Xiaoqiao,Sherwood
> >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> >> > discussions
> >> > > > and
> >> > > > > contributing to this.
> >> > > > >
> >> > > > > *Future Tasks:*
> >> > > > > ============
> >> > > > >
> >> > > > > will cleanup the jira's under this umbrella and contiue to
> work.
> >> > > > >
> >> > > > > Reference:
> >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --Brahma Reddy Battula
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > For additional commands, e-mail:
> >> > > > hdfs-dev-help@hadoop.apache.org
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >> Regards,
> >> Chitta
> >
> >
> >>
>
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
>
Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Xiaoqiao He <xq...@gmail.com>.
+1 (non-binding)
- Try to merge branch HDFS-13891(RBF) to trunk at local and no conflict or
failure.
- Built from merged sources.
- Ran 85 RBF test class at local and result shows: (Tests run: 639,
Failures: 2, Errors: 2, Skipped: 2) - failed tests include
#TestRouterWithSecureStartup and #TestRouterHttpDelegationToken. I don't
think it is blocked issue.
Thanks Brahma for organizing the merge.
On Mon, Jun 10, 2019 at 12:27 PM Ranith Sardar <ra...@huawei.com>
wrote:
> +1 (Non-binding)
>
> -----Original Message-----
> From: Brahma Reddy Battula [mailto:brahma@apache.org]
> Sent: 09 June 2019 19:31
> To: Hadoop Common <co...@hadoop.apache.org>; Hdfs-dev <
> hdfs-dev@hadoop.apache.org>
> Subject: [VOTE] Merge HDFS-13891(RBF) to trunk
>
> Updated mail...
>
> ---------- Forwarded message ---------
> From: Brahma Reddy Battula <br...@apache.org>
> Date: Sun, Jun 9, 2019 at 7:26 PM
> Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
> To: Xiaoqiao He <xq...@gmail.com>
> Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota <
> chitts.hota@gmail.com>, Giovanni Matteo Fumarola <
> giovanni.fumarola@gmail.com>, Hadoop Common <co...@hadoop.apache.org>,
> Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
>
>
> Hi All,
>
> Given the positive response to the discussion thread [1], here is the
> formal vote thread to merge HDFS-13891 in to trunk.
>
> Summary of code changes:
> 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> subproject, there is no impact to hadoop-hdfs and hadoop-common.
> 2. Added Security support for RBF
> 3. Added Missing Client Protocol API's
> 4. Bug Fixes/ Improvments
>
>
> The vote will run for 7 days, ending Sat June 15th. I will start this
> vote with my +1.
>
> Regards,
> Brahma Reddy Battula
>
> 1).
>
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
>
>
> On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
>
> > Thanks Brahma for starting the thread.
> > +1 for merging.
> >
> > He Xiaoqiao
> >
> > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota
> > <ch...@gmail.com>
> > wrote:
> >
> >> Thanks Brahma initiating this.
> >> +1(non-binding) for merge.
> >>
> >> @Uber we have almost all changes specially rbf security in production
> >> for a while now without issues.
> >>
> >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> >> giovanni.fumarola@gmail.com> wrote:
> >>
> >> > +1 on merging.
> >> >
> >> > Thanks Brahma for starting the thread.
> >> >
> >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com>
> wrote:
> >> >
> >> > > Thank you Brahma for pushing this.
> >> > >
> >> > > As you mentioned, we have already taken most of the changes into
> >> > > production.
> >> > > I want to highlight that the main contribution is the addition of
> >> > security.
> >> > > We have been able to test this at a smaller scale (~500 servers
> >> > > and 4
> >> > > subclusters) and the performance is great with our current
> >> > > ZooKeeper deployment.
> >> > > I would also like to highlight that all the changes are
> >> > > constrained to hadoop-hdfs-rbf and there is no differences in
> commons or HDFS.
> >> > >
> >> > > +1 on merging
> >> > >
> >> > > Inigo
> >> > >
> >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka
> >> > > <aa...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Thanks Brahma for starting the discussion.
> >> > > > I'm +1 for merging this.
> >> > > >
> >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
> >> > > > cluster with 2 routers (not in production) and running several
> tests.
> >> > > >
> >> > > > Regards,
> >> > > > Akira
> >> > > >
> >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> >> > brahma@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > Dear Hadoop Developers
> >> > > > >
> >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into
> trunk.
> >> We
> >> > > have
> >> > > > > been working on this feature from last several months.
> >> > > > > This feature work received the contributions from different
> >> > companies.
> >> > > > All
> >> > > > > of the feature development happened smoothly and
> >> > > > > collaboratively
> >> in
> >> > > > JIRAs.
> >> > > > >
> >> > > > > Kindly do take a look at the branch and raise issues/concerns
> >> > > > > that
> >> > need
> >> > > > to
> >> > > > > be addressed before the merge.
> >> > > > >
> >> > > > > *Highlights of HDFS-13891 Branch:*
> >> > > > > =============================
> >> > > > >
> >> > > > > Adding Security to RBF(1)
> >> > > > > Adding Missing Client API's(2) Improvements/Bug Fixing
> >> > > > > Critical - HDFS-13637, HDFS-13834
> >> > > > >
> >> > > > > *Commits:*
> >> > > > > ========
> >> > > > >
> >> > > > > No of JIRAs Resolved: 72
> >> > > > >
> >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> >> > > > >
> >> > > > > *Tested Cluster:*
> >> > > > > =============
> >> > > > >
> >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and
> >> > > > > some
> >> > other
> >> > > > > compaines.
> >> > > > >
> >> > > > > *Uber*: Most changes are running in production @Uber
> >> > > > > including the
> >> > > > critical
> >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> >> Routers.
> >> > > > > Zookeeper as a state store to hold delegation tokens were
> >> > > > > also
> >> stress
> >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> >> > > > >
> >> > > > > *MicroSoft*: Most of these changes are currently running in
> >> > production
> >> > > at
> >> > > > > Microsoft.The security has also been tested in a 500 server
> >> cluster
> >> > > with
> >> > > > 4
> >> > > > > subclsuters. --Inigo Goiri
> >> > > > >
> >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with
> >> > > > > 3 routers.Planning deploy 10K production cluster.
> >> > > > >
> >> > > > > *Contributors:*
> >> > > > > ===========
> >> > > > >
> >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> >> Asanuma,Shubham
> >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
> >> > > > > Li,Gang Li,Surendra Singh Lihore,Ranith Sardar,Ayush
> >> > > > > Saxena,He
> >> > > Xiaoqiao,Sherwood
> >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> >> > discussions
> >> > > > and
> >> > > > > contributing to this.
> >> > > > >
> >> > > > > *Future Tasks:*
> >> > > > > ============
> >> > > > >
> >> > > > > will cleanup the jira's under this umbrella and contiue to
> work.
> >> > > > >
> >> > > > > Reference:
> >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --Brahma Reddy Battula
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > For additional commands, e-mail:
> >> > > > hdfs-dev-help@hadoop.apache.org
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >> Regards,
> >> Chitta
> >
> >
> >>
>
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
>
Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Xiaoqiao He <xq...@gmail.com>.
+1 (non-binding)
- Try to merge branch HDFS-13891(RBF) to trunk at local and no conflict or
failure.
- Built from merged sources.
- Ran 85 RBF test class at local and result shows: (Tests run: 639,
Failures: 2, Errors: 2, Skipped: 2) - failed tests include
#TestRouterWithSecureStartup and #TestRouterHttpDelegationToken. I don't
think it is blocked issue.
Thanks Brahma for organizing the merge.
On Mon, Jun 10, 2019 at 12:27 PM Ranith Sardar <ra...@huawei.com>
wrote:
> +1 (Non-binding)
>
> -----Original Message-----
> From: Brahma Reddy Battula [mailto:brahma@apache.org]
> Sent: 09 June 2019 19:31
> To: Hadoop Common <co...@hadoop.apache.org>; Hdfs-dev <
> hdfs-dev@hadoop.apache.org>
> Subject: [VOTE] Merge HDFS-13891(RBF) to trunk
>
> Updated mail...
>
> ---------- Forwarded message ---------
> From: Brahma Reddy Battula <br...@apache.org>
> Date: Sun, Jun 9, 2019 at 7:26 PM
> Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
> To: Xiaoqiao He <xq...@gmail.com>
> Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota <
> chitts.hota@gmail.com>, Giovanni Matteo Fumarola <
> giovanni.fumarola@gmail.com>, Hadoop Common <co...@hadoop.apache.org>,
> Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
>
>
> Hi All,
>
> Given the positive response to the discussion thread [1], here is the
> formal vote thread to merge HDFS-13891 in to trunk.
>
> Summary of code changes:
> 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> subproject, there is no impact to hadoop-hdfs and hadoop-common.
> 2. Added Security support for RBF
> 3. Added Missing Client Protocol API's
> 4. Bug Fixes/ Improvments
>
>
> The vote will run for 7 days, ending Sat June 15th. I will start this
> vote with my +1.
>
> Regards,
> Brahma Reddy Battula
>
> 1).
>
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
>
>
> On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
>
> > Thanks Brahma for starting the thread.
> > +1 for merging.
> >
> > He Xiaoqiao
> >
> > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota
> > <ch...@gmail.com>
> > wrote:
> >
> >> Thanks Brahma initiating this.
> >> +1(non-binding) for merge.
> >>
> >> @Uber we have almost all changes specially rbf security in production
> >> for a while now without issues.
> >>
> >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> >> giovanni.fumarola@gmail.com> wrote:
> >>
> >> > +1 on merging.
> >> >
> >> > Thanks Brahma for starting the thread.
> >> >
> >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com>
> wrote:
> >> >
> >> > > Thank you Brahma for pushing this.
> >> > >
> >> > > As you mentioned, we have already taken most of the changes into
> >> > > production.
> >> > > I want to highlight that the main contribution is the addition of
> >> > security.
> >> > > We have been able to test this at a smaller scale (~500 servers
> >> > > and 4
> >> > > subclusters) and the performance is great with our current
> >> > > ZooKeeper deployment.
> >> > > I would also like to highlight that all the changes are
> >> > > constrained to hadoop-hdfs-rbf and there is no differences in
> commons or HDFS.
> >> > >
> >> > > +1 on merging
> >> > >
> >> > > Inigo
> >> > >
> >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka
> >> > > <aa...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Thanks Brahma for starting the discussion.
> >> > > > I'm +1 for merging this.
> >> > > >
> >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
> >> > > > cluster with 2 routers (not in production) and running several
> tests.
> >> > > >
> >> > > > Regards,
> >> > > > Akira
> >> > > >
> >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> >> > brahma@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > Dear Hadoop Developers
> >> > > > >
> >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into
> trunk.
> >> We
> >> > > have
> >> > > > > been working on this feature from last several months.
> >> > > > > This feature work received the contributions from different
> >> > companies.
> >> > > > All
> >> > > > > of the feature development happened smoothly and
> >> > > > > collaboratively
> >> in
> >> > > > JIRAs.
> >> > > > >
> >> > > > > Kindly do take a look at the branch and raise issues/concerns
> >> > > > > that
> >> > need
> >> > > > to
> >> > > > > be addressed before the merge.
> >> > > > >
> >> > > > > *Highlights of HDFS-13891 Branch:*
> >> > > > > =============================
> >> > > > >
> >> > > > > Adding Security to RBF(1)
> >> > > > > Adding Missing Client API's(2) Improvements/Bug Fixing
> >> > > > > Critical - HDFS-13637, HDFS-13834
> >> > > > >
> >> > > > > *Commits:*
> >> > > > > ========
> >> > > > >
> >> > > > > No of JIRAs Resolved: 72
> >> > > > >
> >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> >> > > > >
> >> > > > > *Tested Cluster:*
> >> > > > > =============
> >> > > > >
> >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and
> >> > > > > some
> >> > other
> >> > > > > compaines.
> >> > > > >
> >> > > > > *Uber*: Most changes are running in production @Uber
> >> > > > > including the
> >> > > > critical
> >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> >> Routers.
> >> > > > > Zookeeper as a state store to hold delegation tokens were
> >> > > > > also
> >> stress
> >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> >> > > > >
> >> > > > > *MicroSoft*: Most of these changes are currently running in
> >> > production
> >> > > at
> >> > > > > Microsoft.The security has also been tested in a 500 server
> >> cluster
> >> > > with
> >> > > > 4
> >> > > > > subclsuters. --Inigo Goiri
> >> > > > >
> >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with
> >> > > > > 3 routers.Planning deploy 10K production cluster.
> >> > > > >
> >> > > > > *Contributors:*
> >> > > > > ===========
> >> > > > >
> >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> >> Asanuma,Shubham
> >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
> >> > > > > Li,Gang Li,Surendra Singh Lihore,Ranith Sardar,Ayush
> >> > > > > Saxena,He
> >> > > Xiaoqiao,Sherwood
> >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> >> > discussions
> >> > > > and
> >> > > > > contributing to this.
> >> > > > >
> >> > > > > *Future Tasks:*
> >> > > > > ============
> >> > > > >
> >> > > > > will cleanup the jira's under this umbrella and contiue to
> work.
> >> > > > >
> >> > > > > Reference:
> >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --Brahma Reddy Battula
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > For additional commands, e-mail:
> >> > > > hdfs-dev-help@hadoop.apache.org
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >> Regards,
> >> Chitta
> >
> >
> >>
>
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
>
RE: [VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Ranith Sardar <ra...@huawei.com>.
+1 (Non-binding)
-----Original Message-----
From: Brahma Reddy Battula [mailto:brahma@apache.org]
Sent: 09 June 2019 19:31
To: Hadoop Common <co...@hadoop.apache.org>; Hdfs-dev <hd...@hadoop.apache.org>
Subject: [VOTE] Merge HDFS-13891(RBF) to trunk
Updated mail...
---------- Forwarded message ---------
From: Brahma Reddy Battula <br...@apache.org>
Date: Sun, Jun 9, 2019 at 7:26 PM
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
To: Xiaoqiao He <xq...@gmail.com>
Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota < chitts.hota@gmail.com>, Giovanni Matteo Fumarola < giovanni.fumarola@gmail.com>, Hadoop Common <co...@hadoop.apache.org>, Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
Hi All,
Given the positive response to the discussion thread [1], here is the formal vote thread to merge HDFS-13891 in to trunk.
Summary of code changes:
1. Code changes for this branch are done in the hadoop-hdfs-rbf subproject, there is no impact to hadoop-hdfs and hadoop-common.
2. Added Security support for RBF
3. Added Missing Client Protocol API's
4. Bug Fixes/ Improvments
The vote will run for 7 days, ending Sat June 15th. I will start this vote with my +1.
Regards,
Brahma Reddy Battula
1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> Thanks Brahma for starting the thread.
> +1 for merging.
>
> He Xiaoqiao
>
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota
> <ch...@gmail.com>
> wrote:
>
>> Thanks Brahma initiating this.
>> +1(non-binding) for merge.
>>
>> @Uber we have almost all changes specially rbf security in production
>> for a while now without issues.
>>
>> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
>> giovanni.fumarola@gmail.com> wrote:
>>
>> > +1 on merging.
>> >
>> > Thanks Brahma for starting the thread.
>> >
>> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
>> >
>> > > Thank you Brahma for pushing this.
>> > >
>> > > As you mentioned, we have already taken most of the changes into
>> > > production.
>> > > I want to highlight that the main contribution is the addition of
>> > security.
>> > > We have been able to test this at a smaller scale (~500 servers
>> > > and 4
>> > > subclusters) and the performance is great with our current
>> > > ZooKeeper deployment.
>> > > I would also like to highlight that all the changes are
>> > > constrained to hadoop-hdfs-rbf and there is no differences in commons or HDFS.
>> > >
>> > > +1 on merging
>> > >
>> > > Inigo
>> > >
>> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka
>> > > <aa...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks Brahma for starting the discussion.
>> > > > I'm +1 for merging this.
>> > > >
>> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
>> > > > cluster with 2 routers (not in production) and running several tests.
>> > > >
>> > > > Regards,
>> > > > Akira
>> > > >
>> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
>> > brahma@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > Dear Hadoop Developers
>> > > > >
>> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
>> We
>> > > have
>> > > > > been working on this feature from last several months.
>> > > > > This feature work received the contributions from different
>> > companies.
>> > > > All
>> > > > > of the feature development happened smoothly and
>> > > > > collaboratively
>> in
>> > > > JIRAs.
>> > > > >
>> > > > > Kindly do take a look at the branch and raise issues/concerns
>> > > > > that
>> > need
>> > > > to
>> > > > > be addressed before the merge.
>> > > > >
>> > > > > *Highlights of HDFS-13891 Branch:*
>> > > > > =============================
>> > > > >
>> > > > > Adding Security to RBF(1)
>> > > > > Adding Missing Client API's(2) Improvements/Bug Fixing
>> > > > > Critical - HDFS-13637, HDFS-13834
>> > > > >
>> > > > > *Commits:*
>> > > > > ========
>> > > > >
>> > > > > No of JIRAs Resolved: 72
>> > > > >
>> > > > > All this commits are in RBF Module. No changes in hdfs/common.
>> > > > >
>> > > > > *Tested Cluster:*
>> > > > > =============
>> > > > >
>> > > > > Most of these changes verified at Uber,Microsoft,Huawei and
>> > > > > some
>> > other
>> > > > > compaines.
>> > > > >
>> > > > > *Uber*: Most changes are running in production @Uber
>> > > > > including the
>> > > > critical
>> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
>> Routers.
>> > > > > Zookeeper as a state store to hold delegation tokens were
>> > > > > also
>> stress
>> > > > > tested to hold more than 2 Million tokens. --CR Hota
>> > > > >
>> > > > > *MicroSoft*: Most of these changes are currently running in
>> > production
>> > > at
>> > > > > Microsoft.The security has also been tested in a 500 server
>> cluster
>> > > with
>> > > > 4
>> > > > > subclsuters. --Inigo Goiri
>> > > > >
>> > > > > *Huawei* : Deployed all this changes in 20 node cluster with
>> > > > > 3 routers.Planning deploy 10K production cluster.
>> > > > >
>> > > > > *Contributors:*
>> > > > > ===========
>> > > > >
>> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
>> Asanuma,Shubham
>> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
>> > > > > Li,Gang Li,Surendra Singh Lihore,Ranith Sardar,Ayush
>> > > > > Saxena,He
>> > > Xiaoqiao,Sherwood
>> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
>> > discussions
>> > > > and
>> > > > > contributing to this.
>> > > > >
>> > > > > *Future Tasks:*
>> > > > > ============
>> > > > >
>> > > > > will cleanup the jira's under this umbrella and contiue to work.
>> > > > >
>> > > > > Reference:
>> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
>> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --Brahma Reddy Battula
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > For additional commands, e-mail:
>> > > > hdfs-dev-help@hadoop.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>> Regards,
>> Chitta
>
>
>>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
[VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Brahma Reddy Battula <br...@apache.org>.
Updated mail...
---------- Forwarded message ---------
From: Brahma Reddy Battula <br...@apache.org>
Date: Sun, Jun 9, 2019 at 7:26 PM
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
To: Xiaoqiao He <xq...@gmail.com>
Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota <
chitts.hota@gmail.com>, Giovanni Matteo Fumarola <
giovanni.fumarola@gmail.com>, Hadoop Common <co...@hadoop.apache.org>,
Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
Hi All,
Given the positive response to the discussion thread [1], here is the
formal vote thread to merge HDFS-13891 in to trunk.
Summary of code changes:
1. Code changes for this branch are done in the hadoop-hdfs-rbf
subproject, there is no impact to hadoop-hdfs and hadoop-common.
2. Added Security support for RBF
3. Added Missing Client Protocol API's
4. Bug Fixes/ Improvments
The vote will run for 7 days, ending Sat June 15th. I will start this
vote with my +1.
Regards,
Brahma Reddy Battula
1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> Thanks Brahma for starting the thread.
> +1 for merging.
>
> He Xiaoqiao
>
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> wrote:
>
>> Thanks Brahma initiating this.
>> +1(non-binding) for merge.
>>
>> @Uber we have almost all changes specially rbf security in production for
>> a
>> while now without issues.
>>
>> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
>> giovanni.fumarola@gmail.com> wrote:
>>
>> > +1 on merging.
>> >
>> > Thanks Brahma for starting the thread.
>> >
>> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
>> >
>> > > Thank you Brahma for pushing this.
>> > >
>> > > As you mentioned, we have already taken most of the changes into
>> > > production.
>> > > I want to highlight that the main contribution is the addition of
>> > security.
>> > > We have been able to test this at a smaller scale (~500 servers and 4
>> > > subclusters) and the performance is great with our current ZooKeeper
>> > > deployment.
>> > > I would also like to highlight that all the changes are constrained to
>> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
>> > >
>> > > +1 on merging
>> > >
>> > > Inigo
>> > >
>> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks Brahma for starting the discussion.
>> > > > I'm +1 for merging this.
>> > > >
>> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
>> > > > with 2 routers (not in production) and running several tests.
>> > > >
>> > > > Regards,
>> > > > Akira
>> > > >
>> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
>> > brahma@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > Dear Hadoop Developers
>> > > > >
>> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
>> We
>> > > have
>> > > > > been working on this feature from last several months.
>> > > > > This feature work received the contributions from different
>> > companies.
>> > > > All
>> > > > > of the feature development happened smoothly and collaboratively
>> in
>> > > > JIRAs.
>> > > > >
>> > > > > Kindly do take a look at the branch and raise issues/concerns that
>> > need
>> > > > to
>> > > > > be addressed before the merge.
>> > > > >
>> > > > > *Highlights of HDFS-13891 Branch:*
>> > > > > =============================
>> > > > >
>> > > > > Adding Security to RBF(1)
>> > > > > Adding Missing Client API's(2)
>> > > > > Improvements/Bug Fixing
>> > > > > Critical - HDFS-13637, HDFS-13834
>> > > > >
>> > > > > *Commits:*
>> > > > > ========
>> > > > >
>> > > > > No of JIRAs Resolved: 72
>> > > > >
>> > > > > All this commits are in RBF Module. No changes in hdfs/common.
>> > > > >
>> > > > > *Tested Cluster:*
>> > > > > =============
>> > > > >
>> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
>> > other
>> > > > > compaines.
>> > > > >
>> > > > > *Uber*: Most changes are running in production @Uber including the
>> > > > critical
>> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
>> Routers.
>> > > > > Zookeeper as a state store to hold delegation tokens were also
>> stress
>> > > > > tested to hold more than 2 Million tokens. --CR Hota
>> > > > >
>> > > > > *MicroSoft*: Most of these changes are currently running in
>> > production
>> > > at
>> > > > > Microsoft.The security has also been tested in a 500 server
>> cluster
>> > > with
>> > > > 4
>> > > > > subclsuters. --Inigo Goiri
>> > > > >
>> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
>> > > > > routers.Planning deploy 10K production cluster.
>> > > > >
>> > > > > *Contributors:*
>> > > > > ===========
>> > > > >
>> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
>> Asanuma,Shubham
>> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
>> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
>> > > Xiaoqiao,Sherwood
>> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
>> > discussions
>> > > > and
>> > > > > contributing to this.
>> > > > >
>> > > > > *Future Tasks:*
>> > > > > ============
>> > > > >
>> > > > > will cleanup the jira's under this umbrella and contiue to work.
>> > > > >
>> > > > > Reference:
>> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
>> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --Brahma Reddy Battula
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>> Regards,
>> Chitta
>
>
>>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
[VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Brahma Reddy Battula <br...@apache.org>.
Updated mail...
---------- Forwarded message ---------
From: Brahma Reddy Battula <br...@apache.org>
Date: Sun, Jun 9, 2019 at 7:26 PM
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
To: Xiaoqiao He <xq...@gmail.com>
Cc: Akira Ajisaka <aa...@gmail.com>, Chittaranjan Hota <
chitts.hota@gmail.com>, Giovanni Matteo Fumarola <
giovanni.fumarola@gmail.com>, Hadoop Common <co...@hadoop.apache.org>,
Hdfs-dev <hd...@hadoop.apache.org>, Iñigo Goiri <el...@gmail.com>
Hi All,
Given the positive response to the discussion thread [1], here is the
formal vote thread to merge HDFS-13891 in to trunk.
Summary of code changes:
1. Code changes for this branch are done in the hadoop-hdfs-rbf
subproject, there is no impact to hadoop-hdfs and hadoop-common.
2. Added Security support for RBF
3. Added Missing Client Protocol API's
4. Bug Fixes/ Improvments
The vote will run for 7 days, ending Sat June 15th. I will start this
vote with my +1.
Regards,
Brahma Reddy Battula
1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> Thanks Brahma for starting the thread.
> +1 for merging.
>
> He Xiaoqiao
>
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> wrote:
>
>> Thanks Brahma initiating this.
>> +1(non-binding) for merge.
>>
>> @Uber we have almost all changes specially rbf security in production for
>> a
>> while now without issues.
>>
>> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
>> giovanni.fumarola@gmail.com> wrote:
>>
>> > +1 on merging.
>> >
>> > Thanks Brahma for starting the thread.
>> >
>> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
>> >
>> > > Thank you Brahma for pushing this.
>> > >
>> > > As you mentioned, we have already taken most of the changes into
>> > > production.
>> > > I want to highlight that the main contribution is the addition of
>> > security.
>> > > We have been able to test this at a smaller scale (~500 servers and 4
>> > > subclusters) and the performance is great with our current ZooKeeper
>> > > deployment.
>> > > I would also like to highlight that all the changes are constrained to
>> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
>> > >
>> > > +1 on merging
>> > >
>> > > Inigo
>> > >
>> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks Brahma for starting the discussion.
>> > > > I'm +1 for merging this.
>> > > >
>> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
>> > > > with 2 routers (not in production) and running several tests.
>> > > >
>> > > > Regards,
>> > > > Akira
>> > > >
>> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
>> > brahma@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > Dear Hadoop Developers
>> > > > >
>> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
>> We
>> > > have
>> > > > > been working on this feature from last several months.
>> > > > > This feature work received the contributions from different
>> > companies.
>> > > > All
>> > > > > of the feature development happened smoothly and collaboratively
>> in
>> > > > JIRAs.
>> > > > >
>> > > > > Kindly do take a look at the branch and raise issues/concerns that
>> > need
>> > > > to
>> > > > > be addressed before the merge.
>> > > > >
>> > > > > *Highlights of HDFS-13891 Branch:*
>> > > > > =============================
>> > > > >
>> > > > > Adding Security to RBF(1)
>> > > > > Adding Missing Client API's(2)
>> > > > > Improvements/Bug Fixing
>> > > > > Critical - HDFS-13637, HDFS-13834
>> > > > >
>> > > > > *Commits:*
>> > > > > ========
>> > > > >
>> > > > > No of JIRAs Resolved: 72
>> > > > >
>> > > > > All this commits are in RBF Module. No changes in hdfs/common.
>> > > > >
>> > > > > *Tested Cluster:*
>> > > > > =============
>> > > > >
>> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
>> > other
>> > > > > compaines.
>> > > > >
>> > > > > *Uber*: Most changes are running in production @Uber including the
>> > > > critical
>> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
>> Routers.
>> > > > > Zookeeper as a state store to hold delegation tokens were also
>> stress
>> > > > > tested to hold more than 2 Million tokens. --CR Hota
>> > > > >
>> > > > > *MicroSoft*: Most of these changes are currently running in
>> > production
>> > > at
>> > > > > Microsoft.The security has also been tested in a 500 server
>> cluster
>> > > with
>> > > > 4
>> > > > > subclsuters. --Inigo Goiri
>> > > > >
>> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
>> > > > > routers.Planning deploy 10K production cluster.
>> > > > >
>> > > > > *Contributors:*
>> > > > > ===========
>> > > > >
>> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
>> Asanuma,Shubham
>> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
>> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
>> > > Xiaoqiao,Sherwood
>> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
>> > discussions
>> > > > and
>> > > > > contributing to this.
>> > > > >
>> > > > > *Future Tasks:*
>> > > > > ============
>> > > > >
>> > > > > will cleanup the jira's under this umbrella and contiue to work.
>> > > > >
>> > > > > Reference:
>> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
>> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --Brahma Reddy Battula
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>> Regards,
>> Chitta
>
>
>>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula
Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Posted by Akira Ajisaka <aa...@apache.org>.
+1
-Akira
On Sun, Jun 9, 2019 at 10:56 PM Brahma Reddy Battula <br...@apache.org> wrote:
>
> Hi All,
>
> Given the positive response to the discussion thread [1], here is the
> formal vote thread to merge HDFS-13891 in to trunk.
>
> Summary of code changes:
> 1. Code changes for this branch are done in the hadoop-hdfs-rbf
> subproject, there is no impact to hadoop-hdfs and hadoop-common.
> 2. Added Security support for RBF
> 3. Added Missing Client Protocol API's
> 4. Bug Fixes/ Improvments
>
>
> The vote will run for 7 days, ending Sat June 15th. I will start this
> vote with my +1.
>
> Regards,
> Brahma Reddy Battula
>
> 1).
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
>
>
> On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
>
> > Thanks Brahma for starting the thread.
> > +1 for merging.
> >
> > He Xiaoqiao
> >
> > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> > wrote:
> >
> >> Thanks Brahma initiating this.
> >> +1(non-binding) for merge.
> >>
> >> @Uber we have almost all changes specially rbf security in production for
> >> a
> >> while now without issues.
> >>
> >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> >> giovanni.fumarola@gmail.com> wrote:
> >>
> >> > +1 on merging.
> >> >
> >> > Thanks Brahma for starting the thread.
> >> >
> >> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
> >> >
> >> > > Thank you Brahma for pushing this.
> >> > >
> >> > > As you mentioned, we have already taken most of the changes into
> >> > > production.
> >> > > I want to highlight that the main contribution is the addition of
> >> > security.
> >> > > We have been able to test this at a smaller scale (~500 servers and 4
> >> > > subclusters) and the performance is great with our current ZooKeeper
> >> > > deployment.
> >> > > I would also like to highlight that all the changes are constrained to
> >> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
> >> > >
> >> > > +1 on merging
> >> > >
> >> > > Inigo
> >> > >
> >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Thanks Brahma for starting the discussion.
> >> > > > I'm +1 for merging this.
> >> > > >
> >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
> >> > > > with 2 routers (not in production) and running several tests.
> >> > > >
> >> > > > Regards,
> >> > > > Akira
> >> > > >
> >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> >> > brahma@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > Dear Hadoop Developers
> >> > > > >
> >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
> >> We
> >> > > have
> >> > > > > been working on this feature from last several months.
> >> > > > > This feature work received the contributions from different
> >> > companies.
> >> > > > All
> >> > > > > of the feature development happened smoothly and collaboratively
> >> in
> >> > > > JIRAs.
> >> > > > >
> >> > > > > Kindly do take a look at the branch and raise issues/concerns that
> >> > need
> >> > > > to
> >> > > > > be addressed before the merge.
> >> > > > >
> >> > > > > *Highlights of HDFS-13891 Branch:*
> >> > > > > =============================
> >> > > > >
> >> > > > > Adding Security to RBF(1)
> >> > > > > Adding Missing Client API's(2)
> >> > > > > Improvements/Bug Fixing
> >> > > > > Critical - HDFS-13637, HDFS-13834
> >> > > > >
> >> > > > > *Commits:*
> >> > > > > ========
> >> > > > >
> >> > > > > No of JIRAs Resolved: 72
> >> > > > >
> >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> >> > > > >
> >> > > > > *Tested Cluster:*
> >> > > > > =============
> >> > > > >
> >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
> >> > other
> >> > > > > compaines.
> >> > > > >
> >> > > > > *Uber*: Most changes are running in production @Uber including the
> >> > > > critical
> >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> >> Routers.
> >> > > > > Zookeeper as a state store to hold delegation tokens were also
> >> stress
> >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> >> > > > >
> >> > > > > *MicroSoft*: Most of these changes are currently running in
> >> > production
> >> > > at
> >> > > > > Microsoft.The security has also been tested in a 500 server
> >> cluster
> >> > > with
> >> > > > 4
> >> > > > > subclsuters. --Inigo Goiri
> >> > > > >
> >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
> >> > > > > routers.Planning deploy 10K production cluster.
> >> > > > >
> >> > > > > *Contributors:*
> >> > > > > ===========
> >> > > > >
> >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> >> Asanuma,Shubham
> >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
> >> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
> >> > > Xiaoqiao,Sherwood
> >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> >> > discussions
> >> > > > and
> >> > > > > contributing to this.
> >> > > > >
> >> > > > > *Future Tasks:*
> >> > > > > ============
> >> > > > >
> >> > > > > will cleanup the jira's under this umbrella and contiue to work.
> >> > > > >
> >> > > > > Reference:
> >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --Brahma Reddy Battula
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >> Regards,
> >> Chitta
> >>
> >
>
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
[Notice] Merge HDFS-13891(RBF) to trunk
Posted by Brahma Reddy Battula <br...@hotmail.com>.
Rebased HDFS-13891 against trunk and ran the jenkins (1).Only two four testcases are failing which flaky and there is already jira to track this. (HDFS-14461)
Going to merge the branch soon which contains 77 commits,so please bear with us.
1.https://builds.apache.org/job/PreCommit-HDFS-Build/27044/console
________________________________
From: Brahma Reddy Battula <br...@apache.org>
Sent: Sunday, June 9, 2019 9:56 PM
To: Xiaoqiao He
Cc: Akira Ajisaka; Chittaranjan Hota; Giovanni Matteo Fumarola; Hadoop Common; Hdfs-dev; Iñigo Goiri
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Hi All,
Given the positive response to the discussion thread [1], here is the
formal vote thread to merge HDFS-13891 in to trunk.
Summary of code changes:
1. Code changes for this branch are done in the hadoop-hdfs-rbf
subproject, there is no impact to hadoop-hdfs and hadoop-common.
2. Added Security support for RBF
3. Added Missing Client Protocol API's
4. Bug Fixes/ Improvments
The vote will run for 7 days, ending Sat June 15th. I will start this
vote with my +1.
Regards,
Brahma Reddy Battula
1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq...@gmail.com> wrote:
> Thanks Brahma for starting the thread.
> +1 for merging.
>
> He Xiaoqiao
>
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <ch...@gmail.com>
> wrote:
>
>> Thanks Brahma initiating this.
>> +1(non-binding) for merge.
>>
>> @Uber we have almost all changes specially rbf security in production for
>> a
>> while now without issues.
>>
>> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
>> giovanni.fumarola@gmail.com> wrote:
>>
>> > +1 on merging.
>> >
>> > Thanks Brahma for starting the thread.
>> >
>> > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <el...@gmail.com> wrote:
>> >
>> > > Thank you Brahma for pushing this.
>> > >
>> > > As you mentioned, we have already taken most of the changes into
>> > > production.
>> > > I want to highlight that the main contribution is the addition of
>> > security.
>> > > We have been able to test this at a smaller scale (~500 servers and 4
>> > > subclusters) and the performance is great with our current ZooKeeper
>> > > deployment.
>> > > I would also like to highlight that all the changes are constrained to
>> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
>> > >
>> > > +1 on merging
>> > >
>> > > Inigo
>> > >
>> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aa...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks Brahma for starting the discussion.
>> > > > I'm +1 for merging this.
>> > > >
>> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
>> > > > with 2 routers (not in production) and running several tests.
>> > > >
>> > > > Regards,
>> > > > Akira
>> > > >
>> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
>> > brahma@apache.org>
>> > > > wrote:
>> > > > >
>> > > > > Dear Hadoop Developers
>> > > > >
>> > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
>> We
>> > > have
>> > > > > been working on this feature from last several months.
>> > > > > This feature work received the contributions from different
>> > companies.
>> > > > All
>> > > > > of the feature development happened smoothly and collaboratively
>> in
>> > > > JIRAs.
>> > > > >
>> > > > > Kindly do take a look at the branch and raise issues/concerns that
>> > need
>> > > > to
>> > > > > be addressed before the merge.
>> > > > >
>> > > > > *Highlights of HDFS-13891 Branch:*
>> > > > > =============================
>> > > > >
>> > > > > Adding Security to RBF(1)
>> > > > > Adding Missing Client API's(2)
>> > > > > Improvements/Bug Fixing
>> > > > > Critical - HDFS-13637, HDFS-13834
>> > > > >
>> > > > > *Commits:*
>> > > > > ========
>> > > > >
>> > > > > No of JIRAs Resolved: 72
>> > > > >
>> > > > > All this commits are in RBF Module. No changes in hdfs/common.
>> > > > >
>> > > > > *Tested Cluster:*
>> > > > > =============
>> > > > >
>> > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
>> > other
>> > > > > compaines.
>> > > > >
>> > > > > *Uber*: Most changes are running in production @Uber including the
>> > > > critical
>> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
>> Routers.
>> > > > > Zookeeper as a state store to hold delegation tokens were also
>> stress
>> > > > > tested to hold more than 2 Million tokens. --CR Hota
>> > > > >
>> > > > > *MicroSoft*: Most of these changes are currently running in
>> > production
>> > > at
>> > > > > Microsoft.The security has also been tested in a 500 server
>> cluster
>> > > with
>> > > > 4
>> > > > > subclsuters. --Inigo Goiri
>> > > > >
>> > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
>> > > > > routers.Planning deploy 10K production cluster.
>> > > > >
>> > > > > *Contributors:*
>> > > > > ===========
>> > > > >
>> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
>> Asanuma,Shubham
>> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
>> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
>> > > Xiaoqiao,Sherwood
>> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
>> > discussions
>> > > > and
>> > > > > contributing to this.
>> > > > >
>> > > > > *Future Tasks:*
>> > > > > ============
>> > > > >
>> > > > > will cleanup the jira's under this umbrella and contiue to work.
>> > > > >
>> > > > > Reference:
>> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
>> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --Brahma Reddy Battula
>> > > >
>> > > >
>> ---------------------------------------------------------------------
>> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>> Regards,
>> Chitta
>>
>
--
--Brahma Reddy Battula
--
--Brahma Reddy Battula