You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Elek, Marton" <el...@apache.org> on 2019/09/17 09:49:15 UTC
[DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source
tree
TLDR; I propose to move Ozone related code out from Hadoop trunk and
store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
be part of the source tree but with separated release cadence, mainly
because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
During the last Ozone releases this dependency is removed to provide
more stable releases. Instead of using the latest trunk/SNAPSHOT build
from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
As we have no more strict dependency between Hadoop trunk SNAPSHOT and
Ozone trunk I propose to separate the two code base from each other with
creating a new Hadoop git repository (apache/hadoop-ozone.git):
With moving Ozone to a separated git repository:
* It would be easier to contribute and understand the build (as of now
we always need `-f pom.ozone.xml` as a Maven parameter)
* It would be possible to adjust build process without breaking
Hadoop/Ozone builds.
* It would be possible to use different Readme/.asf.yaml/github
template for the Hadoop Ozone and core Hadoop. (For example the current
github template [2] has a link to the contribution guideline [3]. Ozone
has an extended version [4] from this guideline with additional
information.)
* Testing would be more safe as it won't be possible to change core
Hadoop and Hadoop Ozone in the same patch.
* It would be easier to cut branches for Hadoop releases (based on the
original consensus, Ozone should be removed from all the release
branches after creating relase branches from trunk)
What do you think?
Thanks,
Marton
[1]:
https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
[2]:
https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
[3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
[4]:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Arpit Agarwal <aa...@cloudera.com.INVALID>.
+1
> On Sep 17, 2019, at 2:49 AM, Elek, Marton <el...@apache.org> wrote:
>
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be part of the source tree but with separated release cadence, mainly because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide more stable releases. Instead of using the latest trunk/SNAPSHOT build from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone trunk I propose to separate the two code base from each other with creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github template for the Hadoop Ozone and core Hadoop. (For example the current github template [2] has a link to the contribution guideline [3]. Ozone has an extended version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the original consensus, Ozone should be removed from all the release branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]: https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Wanqiang Ji <wa...@gmail.com>.
+1 (non-binding)
BR,
Wanqiang Ji
On Tue, Sep 17, 2019 at 5:48 PM Elek, Marton <el...@apache.org> wrote:
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Arpit Agarwal <aa...@cloudera.com.INVALID>.
+1
> On Sep 17, 2019, at 2:49 AM, Elek, Marton <el...@apache.org> wrote:
>
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be part of the source tree but with separated release cadence, mainly because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide more stable releases. Instead of using the latest trunk/SNAPSHOT build from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone trunk I propose to separate the two code base from each other with creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github template for the Hadoop Ozone and core Hadoop. (For example the current github template [2] has a link to the contribution guideline [3]. Ozone has an extended version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the original consensus, Ozone should be removed from all the release branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]: https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Arpit Agarwal <aa...@cloudera.com.INVALID>.
+1
> On Sep 17, 2019, at 2:49 AM, Elek, Marton <el...@apache.org> wrote:
>
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be part of the source tree but with separated release cadence, mainly because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide more stable releases. Instead of using the latest trunk/SNAPSHOT build from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone trunk I propose to separate the two code base from each other with creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github template for the Hadoop Ozone and core Hadoop. (For example the current github template [2] has a link to the contribution guideline [3]. Ozone has an extended version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the original consensus, Ozone should be removed from all the release branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]: https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Peter Bacsko <pb...@cloudera.com.INVALID>.
+1 (non-binding)
On Fri, Sep 20, 2019 at 8:01 AM Rakesh Radhakrishnan <ra...@apache.org>
wrote:
> +1
>
> Rakesh
>
> On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri <aj...@gmail.com> wrote:
>
> > +1 (binding)
> >
> > Thanks to the Ozone folks for their efforts at maintaining good
> separation
> > with HDFS and common. I took a lot of heat for the unpopular opinion that
> > they should be separate, so I am glad the process has worked out well
> for
> > both codebases. It looks like my concerns were addressed and I appreciate
> > it. It is cool to see the evolution here.
> >
> > Aaron
> >
> >
> > On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > in that case,
> > >
> > > +1 from me (binding)
> > >
> > > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> > >
> > > > > one thing to consider here as you are giving up your ability to
> make
> > > > > changes in hadoop-* modules, including hadoop-common, and their
> > > > > dependencies, in sync with your own code. That goes for filesystem
> > > > contract
> > > > > tests.
> > > > >
> > > > > are you happy with that?
> > > >
> > > >
> > > > Yes. I think we can live with it.
> > > >
> > > > Fortunatelly the Hadoop parts which are used by Ozone (security +
> rpc)
> > > > are stable enough, we didn't need bigger changes until now (small
> > > > patches are already included in 3.1/3.2).
> > > >
> > > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > > worst (best?) case we can try to do more frequent patch releases from
> > > > Hadoop (if required).
> > > >
> > > >
> > > > m.
> > > >
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Peter Bacsko <pb...@cloudera.com.INVALID>.
+1 (non-binding)
On Fri, Sep 20, 2019 at 8:01 AM Rakesh Radhakrishnan <ra...@apache.org>
wrote:
> +1
>
> Rakesh
>
> On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri <aj...@gmail.com> wrote:
>
> > +1 (binding)
> >
> > Thanks to the Ozone folks for their efforts at maintaining good
> separation
> > with HDFS and common. I took a lot of heat for the unpopular opinion that
> > they should be separate, so I am glad the process has worked out well
> for
> > both codebases. It looks like my concerns were addressed and I appreciate
> > it. It is cool to see the evolution here.
> >
> > Aaron
> >
> >
> > On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > in that case,
> > >
> > > +1 from me (binding)
> > >
> > > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> > >
> > > > > one thing to consider here as you are giving up your ability to
> make
> > > > > changes in hadoop-* modules, including hadoop-common, and their
> > > > > dependencies, in sync with your own code. That goes for filesystem
> > > > contract
> > > > > tests.
> > > > >
> > > > > are you happy with that?
> > > >
> > > >
> > > > Yes. I think we can live with it.
> > > >
> > > > Fortunatelly the Hadoop parts which are used by Ozone (security +
> rpc)
> > > > are stable enough, we didn't need bigger changes until now (small
> > > > patches are already included in 3.1/3.2).
> > > >
> > > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > > worst (best?) case we can try to do more frequent patch releases from
> > > > Hadoop (if required).
> > > >
> > > >
> > > > m.
> > > >
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Peter Bacsko <pb...@cloudera.com.INVALID>.
+1 (non-binding)
On Fri, Sep 20, 2019 at 8:01 AM Rakesh Radhakrishnan <ra...@apache.org>
wrote:
> +1
>
> Rakesh
>
> On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri <aj...@gmail.com> wrote:
>
> > +1 (binding)
> >
> > Thanks to the Ozone folks for their efforts at maintaining good
> separation
> > with HDFS and common. I took a lot of heat for the unpopular opinion that
> > they should be separate, so I am glad the process has worked out well
> for
> > both codebases. It looks like my concerns were addressed and I appreciate
> > it. It is cool to see the evolution here.
> >
> > Aaron
> >
> >
> > On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > in that case,
> > >
> > > +1 from me (binding)
> > >
> > > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> > >
> > > > > one thing to consider here as you are giving up your ability to
> make
> > > > > changes in hadoop-* modules, including hadoop-common, and their
> > > > > dependencies, in sync with your own code. That goes for filesystem
> > > > contract
> > > > > tests.
> > > > >
> > > > > are you happy with that?
> > > >
> > > >
> > > > Yes. I think we can live with it.
> > > >
> > > > Fortunatelly the Hadoop parts which are used by Ozone (security +
> rpc)
> > > > are stable enough, we didn't need bigger changes until now (small
> > > > patches are already included in 3.1/3.2).
> > > >
> > > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > > worst (best?) case we can try to do more frequent patch releases from
> > > > Hadoop (if required).
> > > >
> > > >
> > > > m.
> > > >
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Peter Bacsko <pb...@cloudera.com.INVALID>.
+1 (non-binding)
On Fri, Sep 20, 2019 at 8:01 AM Rakesh Radhakrishnan <ra...@apache.org>
wrote:
> +1
>
> Rakesh
>
> On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri <aj...@gmail.com> wrote:
>
> > +1 (binding)
> >
> > Thanks to the Ozone folks for their efforts at maintaining good
> separation
> > with HDFS and common. I took a lot of heat for the unpopular opinion that
> > they should be separate, so I am glad the process has worked out well
> for
> > both codebases. It looks like my concerns were addressed and I appreciate
> > it. It is cool to see the evolution here.
> >
> > Aaron
> >
> >
> > On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > in that case,
> > >
> > > +1 from me (binding)
> > >
> > > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> > >
> > > > > one thing to consider here as you are giving up your ability to
> make
> > > > > changes in hadoop-* modules, including hadoop-common, and their
> > > > > dependencies, in sync with your own code. That goes for filesystem
> > > > contract
> > > > > tests.
> > > > >
> > > > > are you happy with that?
> > > >
> > > >
> > > > Yes. I think we can live with it.
> > > >
> > > > Fortunatelly the Hadoop parts which are used by Ozone (security +
> rpc)
> > > > are stable enough, we didn't need bigger changes until now (small
> > > > patches are already included in 3.1/3.2).
> > > >
> > > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > > worst (best?) case we can try to do more frequent patch releases from
> > > > Hadoop (if required).
> > > >
> > > >
> > > > m.
> > > >
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
+1
Rakesh
On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri <aj...@gmail.com> wrote:
> +1 (binding)
>
> Thanks to the Ozone folks for their efforts at maintaining good separation
> with HDFS and common. I took a lot of heat for the unpopular opinion that
> they should be separate, so I am glad the process has worked out well for
> both codebases. It looks like my concerns were addressed and I appreciate
> it. It is cool to see the evolution here.
>
> Aaron
>
>
> On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > in that case,
> >
> > +1 from me (binding)
> >
> > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> >
> > > > one thing to consider here as you are giving up your ability to make
> > > > changes in hadoop-* modules, including hadoop-common, and their
> > > > dependencies, in sync with your own code. That goes for filesystem
> > > contract
> > > > tests.
> > > >
> > > > are you happy with that?
> > >
> > >
> > > Yes. I think we can live with it.
> > >
> > > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > > are stable enough, we didn't need bigger changes until now (small
> > > patches are already included in 3.1/3.2).
> > >
> > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > worst (best?) case we can try to do more frequent patch releases from
> > > Hadoop (if required).
> > >
> > >
> > > m.
> > >
> > >
> > >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
+1
Rakesh
On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri <aj...@gmail.com> wrote:
> +1 (binding)
>
> Thanks to the Ozone folks for their efforts at maintaining good separation
> with HDFS and common. I took a lot of heat for the unpopular opinion that
> they should be separate, so I am glad the process has worked out well for
> both codebases. It looks like my concerns were addressed and I appreciate
> it. It is cool to see the evolution here.
>
> Aaron
>
>
> On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > in that case,
> >
> > +1 from me (binding)
> >
> > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> >
> > > > one thing to consider here as you are giving up your ability to make
> > > > changes in hadoop-* modules, including hadoop-common, and their
> > > > dependencies, in sync with your own code. That goes for filesystem
> > > contract
> > > > tests.
> > > >
> > > > are you happy with that?
> > >
> > >
> > > Yes. I think we can live with it.
> > >
> > > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > > are stable enough, we didn't need bigger changes until now (small
> > > patches are already included in 3.1/3.2).
> > >
> > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > worst (best?) case we can try to do more frequent patch releases from
> > > Hadoop (if required).
> > >
> > >
> > > m.
> > >
> > >
> > >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
+1
Rakesh
On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri <aj...@gmail.com> wrote:
> +1 (binding)
>
> Thanks to the Ozone folks for their efforts at maintaining good separation
> with HDFS and common. I took a lot of heat for the unpopular opinion that
> they should be separate, so I am glad the process has worked out well for
> both codebases. It looks like my concerns were addressed and I appreciate
> it. It is cool to see the evolution here.
>
> Aaron
>
>
> On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > in that case,
> >
> > +1 from me (binding)
> >
> > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> >
> > > > one thing to consider here as you are giving up your ability to make
> > > > changes in hadoop-* modules, including hadoop-common, and their
> > > > dependencies, in sync with your own code. That goes for filesystem
> > > contract
> > > > tests.
> > > >
> > > > are you happy with that?
> > >
> > >
> > > Yes. I think we can live with it.
> > >
> > > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > > are stable enough, we didn't need bigger changes until now (small
> > > patches are already included in 3.1/3.2).
> > >
> > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > worst (best?) case we can try to do more frequent patch releases from
> > > Hadoop (if required).
> > >
> > >
> > > m.
> > >
> > >
> > >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
+1
Rakesh
On Fri, Sep 20, 2019 at 12:29 AM Aaron Fabbri <aj...@gmail.com> wrote:
> +1 (binding)
>
> Thanks to the Ozone folks for their efforts at maintaining good separation
> with HDFS and common. I took a lot of heat for the unpopular opinion that
> they should be separate, so I am glad the process has worked out well for
> both codebases. It looks like my concerns were addressed and I appreciate
> it. It is cool to see the evolution here.
>
> Aaron
>
>
> On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > in that case,
> >
> > +1 from me (binding)
> >
> > On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> >
> > > > one thing to consider here as you are giving up your ability to make
> > > > changes in hadoop-* modules, including hadoop-common, and their
> > > > dependencies, in sync with your own code. That goes for filesystem
> > > contract
> > > > tests.
> > > >
> > > > are you happy with that?
> > >
> > >
> > > Yes. I think we can live with it.
> > >
> > > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > > are stable enough, we didn't need bigger changes until now (small
> > > patches are already included in 3.1/3.2).
> > >
> > > I think it's better to use released Hadoop bits in Ozone anyway, and
> > > worst (best?) case we can try to do more frequent patch releases from
> > > Hadoop (if required).
> > >
> > >
> > > m.
> > >
> > >
> > >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Aaron Fabbri <aj...@gmail.com>.
+1 (binding)
Thanks to the Ozone folks for their efforts at maintaining good separation
with HDFS and common. I took a lot of heat for the unpopular opinion that
they should be separate, so I am glad the process has worked out well for
both codebases. It looks like my concerns were addressed and I appreciate
it. It is cool to see the evolution here.
Aaron
On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> in that case,
>
> +1 from me (binding)
>
> On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
>
> > > one thing to consider here as you are giving up your ability to make
> > > changes in hadoop-* modules, including hadoop-common, and their
> > > dependencies, in sync with your own code. That goes for filesystem
> > contract
> > > tests.
> > >
> > > are you happy with that?
> >
> >
> > Yes. I think we can live with it.
> >
> > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > are stable enough, we didn't need bigger changes until now (small
> > patches are already included in 3.1/3.2).
> >
> > I think it's better to use released Hadoop bits in Ozone anyway, and
> > worst (best?) case we can try to do more frequent patch releases from
> > Hadoop (if required).
> >
> >
> > m.
> >
> >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Aaron Fabbri <aj...@gmail.com>.
+1 (binding)
Thanks to the Ozone folks for their efforts at maintaining good separation
with HDFS and common. I took a lot of heat for the unpopular opinion that
they should be separate, so I am glad the process has worked out well for
both codebases. It looks like my concerns were addressed and I appreciate
it. It is cool to see the evolution here.
Aaron
On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> in that case,
>
> +1 from me (binding)
>
> On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
>
> > > one thing to consider here as you are giving up your ability to make
> > > changes in hadoop-* modules, including hadoop-common, and their
> > > dependencies, in sync with your own code. That goes for filesystem
> > contract
> > > tests.
> > >
> > > are you happy with that?
> >
> >
> > Yes. I think we can live with it.
> >
> > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > are stable enough, we didn't need bigger changes until now (small
> > patches are already included in 3.1/3.2).
> >
> > I think it's better to use released Hadoop bits in Ozone anyway, and
> > worst (best?) case we can try to do more frequent patch releases from
> > Hadoop (if required).
> >
> >
> > m.
> >
> >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Aaron Fabbri <aj...@gmail.com>.
+1 (binding)
Thanks to the Ozone folks for their efforts at maintaining good separation
with HDFS and common. I took a lot of heat for the unpopular opinion that
they should be separate, so I am glad the process has worked out well for
both codebases. It looks like my concerns were addressed and I appreciate
it. It is cool to see the evolution here.
Aaron
On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> in that case,
>
> +1 from me (binding)
>
> On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
>
> > > one thing to consider here as you are giving up your ability to make
> > > changes in hadoop-* modules, including hadoop-common, and their
> > > dependencies, in sync with your own code. That goes for filesystem
> > contract
> > > tests.
> > >
> > > are you happy with that?
> >
> >
> > Yes. I think we can live with it.
> >
> > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > are stable enough, we didn't need bigger changes until now (small
> > patches are already included in 3.1/3.2).
> >
> > I think it's better to use released Hadoop bits in Ozone anyway, and
> > worst (best?) case we can try to do more frequent patch releases from
> > Hadoop (if required).
> >
> >
> > m.
> >
> >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Aaron Fabbri <aj...@gmail.com>.
+1 (binding)
Thanks to the Ozone folks for their efforts at maintaining good separation
with HDFS and common. I took a lot of heat for the unpopular opinion that
they should be separate, so I am glad the process has worked out well for
both codebases. It looks like my concerns were addressed and I appreciate
it. It is cool to see the evolution here.
Aaron
On Thu, Sep 19, 2019 at 3:37 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> in that case,
>
> +1 from me (binding)
>
> On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
>
> > > one thing to consider here as you are giving up your ability to make
> > > changes in hadoop-* modules, including hadoop-common, and their
> > > dependencies, in sync with your own code. That goes for filesystem
> > contract
> > > tests.
> > >
> > > are you happy with that?
> >
> >
> > Yes. I think we can live with it.
> >
> > Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> > are stable enough, we didn't need bigger changes until now (small
> > patches are already included in 3.1/3.2).
> >
> > I think it's better to use released Hadoop bits in Ozone anyway, and
> > worst (best?) case we can try to do more frequent patch releases from
> > Hadoop (if required).
> >
> >
> > m.
> >
> >
> >
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
in that case,
+1 from me (binding)
On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> > one thing to consider here as you are giving up your ability to make
> > changes in hadoop-* modules, including hadoop-common, and their
> > dependencies, in sync with your own code. That goes for filesystem
> contract
> > tests.
> >
> > are you happy with that?
>
>
> Yes. I think we can live with it.
>
> Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> are stable enough, we didn't need bigger changes until now (small
> patches are already included in 3.1/3.2).
>
> I think it's better to use released Hadoop bits in Ozone anyway, and
> worst (best?) case we can try to do more frequent patch releases from
> Hadoop (if required).
>
>
> m.
>
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
in that case,
+1 from me (binding)
On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> > one thing to consider here as you are giving up your ability to make
> > changes in hadoop-* modules, including hadoop-common, and their
> > dependencies, in sync with your own code. That goes for filesystem
> contract
> > tests.
> >
> > are you happy with that?
>
>
> Yes. I think we can live with it.
>
> Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> are stable enough, we didn't need bigger changes until now (small
> patches are already included in 3.1/3.2).
>
> I think it's better to use released Hadoop bits in Ozone anyway, and
> worst (best?) case we can try to do more frequent patch releases from
> Hadoop (if required).
>
>
> m.
>
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
in that case,
+1 from me (binding)
On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> > one thing to consider here as you are giving up your ability to make
> > changes in hadoop-* modules, including hadoop-common, and their
> > dependencies, in sync with your own code. That goes for filesystem
> contract
> > tests.
> >
> > are you happy with that?
>
>
> Yes. I think we can live with it.
>
> Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> are stable enough, we didn't need bigger changes until now (small
> patches are already included in 3.1/3.2).
>
> I think it's better to use released Hadoop bits in Ozone anyway, and
> worst (best?) case we can try to do more frequent patch releases from
> Hadoop (if required).
>
>
> m.
>
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
in that case,
+1 from me (binding)
On Wed, Sep 18, 2019 at 4:33 PM Elek, Marton <el...@apache.org> wrote:
> > one thing to consider here as you are giving up your ability to make
> > changes in hadoop-* modules, including hadoop-common, and their
> > dependencies, in sync with your own code. That goes for filesystem
> contract
> > tests.
> >
> > are you happy with that?
>
>
> Yes. I think we can live with it.
>
> Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
> are stable enough, we didn't need bigger changes until now (small
> patches are already included in 3.1/3.2).
>
> I think it's better to use released Hadoop bits in Ozone anyway, and
> worst (best?) case we can try to do more frequent patch releases from
> Hadoop (if required).
>
>
> m.
>
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by "Elek, Marton" <el...@apache.org>.
> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem
contract
> tests.
>
> are you happy with that?
Yes. I think we can live with it.
Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
are stable enough, we didn't need bigger changes until now (small
patches are already included in 3.1/3.2).
I think it's better to use released Hadoop bits in Ozone anyway, and
worst (best?) case we can try to do more frequent patch releases from
Hadoop (if required).
m.
On 9/18/19 12:06 PM, Steve Loughran wrote:
> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem contract
> tests.
>
> are you happy with that?
>
> On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton <el...@apache.org> wrote:
>
>>
>>
>> TLDR; I propose to move Ozone related code out from Hadoop trunk and
>> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>>
>>
>>
>>
>> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
>> be part of the source tree but with separated release cadence, mainly
>> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>>
>> During the last Ozone releases this dependency is removed to provide
>> more stable releases. Instead of using the latest trunk/SNAPSHOT build
>> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>>
>> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
>> Ozone trunk I propose to separate the two code base from each other with
>> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>>
>> With moving Ozone to a separated git repository:
>>
>> * It would be easier to contribute and understand the build (as of now
>> we always need `-f pom.ozone.xml` as a Maven parameter)
>> * It would be possible to adjust build process without breaking
>> Hadoop/Ozone builds.
>> * It would be possible to use different Readme/.asf.yaml/github
>> template for the Hadoop Ozone and core Hadoop. (For example the current
>> github template [2] has a link to the contribution guideline [3]. Ozone
>> has an extended version [4] from this guideline with additional
>> information.)
>> * Testing would be more safe as it won't be possible to change core
>> Hadoop and Hadoop Ozone in the same patch.
>> * It would be easier to cut branches for Hadoop releases (based on the
>> original consensus, Ozone should be removed from all the release
>> branches after creating relase branches from trunk)
>>
>>
>> What do you think?
>>
>> Thanks,
>> Marton
>>
>> [1]:
>>
>> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
>> [2]:
>>
>> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
>> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
>> [4]:
>>
>> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by "Elek, Marton" <el...@apache.org>.
> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem
contract
> tests.
>
> are you happy with that?
Yes. I think we can live with it.
Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
are stable enough, we didn't need bigger changes until now (small
patches are already included in 3.1/3.2).
I think it's better to use released Hadoop bits in Ozone anyway, and
worst (best?) case we can try to do more frequent patch releases from
Hadoop (if required).
m.
On 9/18/19 12:06 PM, Steve Loughran wrote:
> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem contract
> tests.
>
> are you happy with that?
>
> On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton <el...@apache.org> wrote:
>
>>
>>
>> TLDR; I propose to move Ozone related code out from Hadoop trunk and
>> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>>
>>
>>
>>
>> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
>> be part of the source tree but with separated release cadence, mainly
>> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>>
>> During the last Ozone releases this dependency is removed to provide
>> more stable releases. Instead of using the latest trunk/SNAPSHOT build
>> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>>
>> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
>> Ozone trunk I propose to separate the two code base from each other with
>> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>>
>> With moving Ozone to a separated git repository:
>>
>> * It would be easier to contribute and understand the build (as of now
>> we always need `-f pom.ozone.xml` as a Maven parameter)
>> * It would be possible to adjust build process without breaking
>> Hadoop/Ozone builds.
>> * It would be possible to use different Readme/.asf.yaml/github
>> template for the Hadoop Ozone and core Hadoop. (For example the current
>> github template [2] has a link to the contribution guideline [3]. Ozone
>> has an extended version [4] from this guideline with additional
>> information.)
>> * Testing would be more safe as it won't be possible to change core
>> Hadoop and Hadoop Ozone in the same patch.
>> * It would be easier to cut branches for Hadoop releases (based on the
>> original consensus, Ozone should be removed from all the release
>> branches after creating relase branches from trunk)
>>
>>
>> What do you think?
>>
>> Thanks,
>> Marton
>>
>> [1]:
>>
>> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
>> [2]:
>>
>> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
>> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
>> [4]:
>>
>> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by "Elek, Marton" <el...@apache.org>.
> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem
contract
> tests.
>
> are you happy with that?
Yes. I think we can live with it.
Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
are stable enough, we didn't need bigger changes until now (small
patches are already included in 3.1/3.2).
I think it's better to use released Hadoop bits in Ozone anyway, and
worst (best?) case we can try to do more frequent patch releases from
Hadoop (if required).
m.
On 9/18/19 12:06 PM, Steve Loughran wrote:
> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem contract
> tests.
>
> are you happy with that?
>
> On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton <el...@apache.org> wrote:
>
>>
>>
>> TLDR; I propose to move Ozone related code out from Hadoop trunk and
>> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>>
>>
>>
>>
>> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
>> be part of the source tree but with separated release cadence, mainly
>> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>>
>> During the last Ozone releases this dependency is removed to provide
>> more stable releases. Instead of using the latest trunk/SNAPSHOT build
>> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>>
>> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
>> Ozone trunk I propose to separate the two code base from each other with
>> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>>
>> With moving Ozone to a separated git repository:
>>
>> * It would be easier to contribute and understand the build (as of now
>> we always need `-f pom.ozone.xml` as a Maven parameter)
>> * It would be possible to adjust build process without breaking
>> Hadoop/Ozone builds.
>> * It would be possible to use different Readme/.asf.yaml/github
>> template for the Hadoop Ozone and core Hadoop. (For example the current
>> github template [2] has a link to the contribution guideline [3]. Ozone
>> has an extended version [4] from this guideline with additional
>> information.)
>> * Testing would be more safe as it won't be possible to change core
>> Hadoop and Hadoop Ozone in the same patch.
>> * It would be easier to cut branches for Hadoop releases (based on the
>> original consensus, Ozone should be removed from all the release
>> branches after creating relase branches from trunk)
>>
>>
>> What do you think?
>>
>> Thanks,
>> Marton
>>
>> [1]:
>>
>> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
>> [2]:
>>
>> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
>> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
>> [4]:
>>
>> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by "Elek, Marton" <el...@apache.org>.
> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem
contract
> tests.
>
> are you happy with that?
Yes. I think we can live with it.
Fortunatelly the Hadoop parts which are used by Ozone (security + rpc)
are stable enough, we didn't need bigger changes until now (small
patches are already included in 3.1/3.2).
I think it's better to use released Hadoop bits in Ozone anyway, and
worst (best?) case we can try to do more frequent patch releases from
Hadoop (if required).
m.
On 9/18/19 12:06 PM, Steve Loughran wrote:
> one thing to consider here as you are giving up your ability to make
> changes in hadoop-* modules, including hadoop-common, and their
> dependencies, in sync with your own code. That goes for filesystem contract
> tests.
>
> are you happy with that?
>
> On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton <el...@apache.org> wrote:
>
>>
>>
>> TLDR; I propose to move Ozone related code out from Hadoop trunk and
>> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>>
>>
>>
>>
>> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
>> be part of the source tree but with separated release cadence, mainly
>> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>>
>> During the last Ozone releases this dependency is removed to provide
>> more stable releases. Instead of using the latest trunk/SNAPSHOT build
>> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>>
>> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
>> Ozone trunk I propose to separate the two code base from each other with
>> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>>
>> With moving Ozone to a separated git repository:
>>
>> * It would be easier to contribute and understand the build (as of now
>> we always need `-f pom.ozone.xml` as a Maven parameter)
>> * It would be possible to adjust build process without breaking
>> Hadoop/Ozone builds.
>> * It would be possible to use different Readme/.asf.yaml/github
>> template for the Hadoop Ozone and core Hadoop. (For example the current
>> github template [2] has a link to the contribution guideline [3]. Ozone
>> has an extended version [4] from this guideline with additional
>> information.)
>> * Testing would be more safe as it won't be possible to change core
>> Hadoop and Hadoop Ozone in the same patch.
>> * It would be easier to cut branches for Hadoop releases (based on the
>> original consensus, Ozone should be removed from all the release
>> branches after creating relase branches from trunk)
>>
>>
>> What do you think?
>>
>> Thanks,
>> Marton
>>
>> [1]:
>>
>> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
>> [2]:
>>
>> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
>> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
>> [4]:
>>
>> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
one thing to consider here as you are giving up your ability to make
changes in hadoop-* modules, including hadoop-common, and their
dependencies, in sync with your own code. That goes for filesystem contract
tests.
are you happy with that?
On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton <el...@apache.org> wrote:
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by 俊平堵 <ju...@apache.org>.
+1.
Thanks,
Junping
Elek, Marton <el...@apache.org> 于2019年9月17日周二 下午5:48写道：
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by 俊平堵 <ju...@apache.org>.
+1.
Thanks,
Junping
Elek, Marton <el...@apache.org> 于2019年9月17日周二 下午5:48写道：
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by 俊平堵 <ju...@apache.org>.
+1.
Thanks,
Junping
Elek, Marton <el...@apache.org> 于2019年9月17日周二 下午5:48写道：
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree
Posted by Anu Engineer <ae...@cloudera.com.INVALID>.
+1
—Anu
> On Sep 17, 2019, at 2:49 AM, Elek, Marton <el...@apache.org> wrote:
>
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be part of the source tree but with separated release cadence, mainly because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide more stable releases. Instead of using the latest trunk/SNAPSHOT build from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone trunk I propose to separate the two code base from each other with creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github template for the Hadoop Ozone and core Hadoop. (For example the current github template [2] has a link to the contribution guideline [3]. Ozone has an extended version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the original consensus, Ozone should be removed from all the release branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]: https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Wanqiang Ji <wa...@gmail.com>.
+1 (non-binding)
BR,
Wanqiang Ji
On Tue, Sep 17, 2019 at 5:48 PM Elek, Marton <el...@apache.org> wrote:
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree
Posted by Anu Engineer <ae...@cloudera.com.INVALID>.
+1
—Anu
> On Sep 17, 2019, at 2:49 AM, Elek, Marton <el...@apache.org> wrote:
>
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be part of the source tree but with separated release cadence, mainly because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide more stable releases. Instead of using the latest trunk/SNAPSHOT build from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone trunk I propose to separate the two code base from each other with creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github template for the Hadoop Ozone and core Hadoop. (For example the current github template [2] has a link to the contribution guideline [3]. Ozone has an extended version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the original consensus, Ozone should be removed from all the release branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]: https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Arpit Agarwal <aa...@cloudera.com.INVALID>.
+1
> On Sep 17, 2019, at 2:49 AM, Elek, Marton <el...@apache.org> wrote:
>
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be part of the source tree but with separated release cadence, mainly because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide more stable releases. Instead of using the latest trunk/SNAPSHOT build from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone trunk I propose to separate the two code base from each other with creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github template for the Hadoop Ozone and core Hadoop. (For example the current github template [2] has a link to the contribution guideline [3]. Ozone has an extended version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the original consensus, Ozone should be removed from all the release branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]: https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
one thing to consider here as you are giving up your ability to make
changes in hadoop-* modules, including hadoop-common, and their
dependencies, in sync with your own code. That goes for filesystem contract
tests.
are you happy with that?
On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton <el...@apache.org> wrote:
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree
Posted by Anu Engineer <ae...@cloudera.com.INVALID>.
+1
—Anu
> On Sep 17, 2019, at 2:49 AM, Elek, Marton <el...@apache.org> wrote:
>
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be part of the source tree but with separated release cadence, mainly because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide more stable releases. Instead of using the latest trunk/SNAPSHOT build from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone trunk I propose to separate the two code base from each other with creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github template for the Hadoop Ozone and core Hadoop. (For example the current github template [2] has a link to the contribution guideline [3]. Ozone has an extended version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the original consensus, Ozone should be removed from all the release branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]: https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
one thing to consider here as you are giving up your ability to make
changes in hadoop-* modules, including hadoop-common, and their
dependencies, in sync with your own code. That goes for filesystem contract
tests.
are you happy with that?
On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton <el...@apache.org> wrote:
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by 俊平堵 <ju...@apache.org>.
+1.
Thanks,
Junping
Elek, Marton <el...@apache.org> 于2019年9月17日周二 下午5:48写道：
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
one thing to consider here as you are giving up your ability to make
changes in hadoop-* modules, including hadoop-common, and their
dependencies, in sync with your own code. That goes for filesystem contract
tests.
are you happy with that?
On Tue, Sep 17, 2019 at 10:48 AM Elek, Marton <el...@apache.org> wrote:
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Wanqiang Ji <wa...@gmail.com>.
+1 (non-binding)
BR,
Wanqiang Ji
On Tue, Sep 17, 2019 at 5:48 PM Elek, Marton <el...@apache.org> wrote:
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk
source tree
Posted by Wanqiang Ji <wa...@gmail.com>.
+1 (non-binding)
BR,
Wanqiang Ji
On Tue, Sep 17, 2019 at 5:48 PM Elek, Marton <el...@apache.org> wrote:
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and
> store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to
> be part of the source tree but with separated release cadence, mainly
> because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide
> more stable releases. Instead of using the latest trunk/SNAPSHOT build
> from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and
> Ozone trunk I propose to separate the two code base from each other with
> creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now
> we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking
> Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github
> template for the Hadoop Ozone and core Hadoop. (For example the current
> github template [2] has a link to the contribution guideline [3]. Ozone
> has an extended version [4] from this guideline with additional
> information.)
> * Testing would be more safe as it won't be possible to change core
> Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the
> original consensus, Ozone should be removed from all the release
> branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]:
>
> https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]:
>
> https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]:
>
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Separate Hadoop Core trunk and Hadoop Ozone trunk source tree
Posted by Anu Engineer <ae...@cloudera.com.INVALID>.
+1
—Anu
> On Sep 17, 2019, at 2:49 AM, Elek, Marton <el...@apache.org> wrote:
>
>
>
> TLDR; I propose to move Ozone related code out from Hadoop trunk and store it in a separated *Hadoop* git repository apache/hadoop-ozone.git
>
>
>
>
> When Ozone was adopted as a new Hadoop subproject it was proposed[1] to be part of the source tree but with separated release cadence, mainly because it had the hadoop-trunk/SNAPSHOT as compile time dependency.
>
> During the last Ozone releases this dependency is removed to provide more stable releases. Instead of using the latest trunk/SNAPSHOT build from Hadoop, Ozone uses the latest stable Hadoop (3.2.0 as of now).
>
> As we have no more strict dependency between Hadoop trunk SNAPSHOT and Ozone trunk I propose to separate the two code base from each other with creating a new Hadoop git repository (apache/hadoop-ozone.git):
>
> With moving Ozone to a separated git repository:
>
> * It would be easier to contribute and understand the build (as of now we always need `-f pom.ozone.xml` as a Maven parameter)
> * It would be possible to adjust build process without breaking Hadoop/Ozone builds.
> * It would be possible to use different Readme/.asf.yaml/github template for the Hadoop Ozone and core Hadoop. (For example the current github template [2] has a link to the contribution guideline [3]. Ozone has an extended version [4] from this guideline with additional information.)
> * Testing would be more safe as it won't be possible to change core Hadoop and Hadoop Ozone in the same patch.
> * It would be easier to cut branches for Hadoop releases (based on the original consensus, Ozone should be removed from all the release branches after creating relase branches from trunk)
>
>
> What do you think?
>
> Thanks,
> Marton
>
> [1]: https://lists.apache.org/thread.html/c85e5263dcc0ca1d13cbbe3bcfb53236784a39111b8c353f60582eb4@%3Chdfs-dev.hadoop.apache.org%3E
> [2]: https://github.com/apache/hadoop/blob/trunk/.github/pull_request_template.md
> [3]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
> [4]: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute+to+Ozone
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org