You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Stephan Ewen <se...@apache.org> on 2016/12/02 16:37:49 UTC
[DISCUSS] Merging the FLIP-6 feature branch into the Master branch
Hi all!
I want to start a discussion about merging the FLIP-6 feature branch into
the master branch.
The feature branch implements the following:
- A new RPC abstraction
- A new High-Availability Services Abstraction
- New JobManager / TaskManager / ResourceManager with dynamic slot
allocation
- Re-designed runners to instantiate them
- A new MiniCluster
The feature branch still needs quite a bit of work, but it does not
interfere with the current state of the master branch. All new components
are implemented as separate components with separate tests.
The branch builds fully, all tests pass, and when setting up Flink by any
of the means supported in the Master, the same code is used and none of the
feature branch code is active.
At the same time, it becomes harder for the feature branch to chase the
master branch.
Also, the feature branch starts to contain cleanups that are valuable in
the master branch as well, for example around reusable parts like the "high
availability services".
*My suggestion is the following:*
- Let's merge the feature branch in the near future, i.e., quite soon.
- When we fork off the "release-1.2" branch, we delete the new components
form that branch. That way we avoid having seemingly dead code in the
source release.
- The feature branch classes are mostly in some subpackages, so it should
be quite straight forward to remove them.
What do you think about this?
Greetings,
Stephan
Re: [DISCUSS] Merging the FLIP-6 feature branch into the Master branch
Posted by Till Rohrmann <tr...@apache.org>.
I agree with Stephan. +1 for merging it after forking 1.2 off and trying to
do this in the near future.
Cheers,
Till
On Fri, Dec 9, 2016 at 11:50 AM, Stephan Ewen <se...@apache.org> wrote:
> I would be +1 to merge the FLIP-6 branch to the master branch after the 1.2
> branch is forked off, if we manage to do that in a timely fashion.
>
> Would actually be safer that way...
>
> On Tue, Dec 6, 2016 at 8:49 PM, Robert Metzger <rm...@apache.org>
> wrote:
>
> > I've reactivated the 1.2 release thread and the actual list of release
> > blockers is quite short as it seems.
> > We could actually manage to fork off a branch for the 1.2 release next
> week
> > (this really depends on how fast we get the blockers down)
> > Would it be okay for the people working on "flip-6" to wait until next
> week
> > to make the final decision here?
> >
> > On Fri, Dec 2, 2016 at 7:18 PM, Stephan Ewen <se...@apache.org> wrote:
> >
> > > Hi Greg!
> > >
> > > Yes, originally I was arguing to merge Flip-6 after forking off the 1.2
> > > branch. That would be the preferable solution, but since it seems that
> > the
> > > 1.2 branch may still take a few weeks, I was thinking that we may want
> to
> > > do that earlier.
> > >
> > > Many flip-6 contributors would like to be active around the Christmas
> > time,
> > > and I almost expect the 1.2 branch will not be forked off by then.
> > >
> > > Stephan
> > >
> > >
> > > On Fri, Dec 2, 2016 at 6:16 PM, Greg Hogan <co...@greghogan.com> wrote:
> > >
> > > > Hi Stephan,
> > > >
> > > > How soon are you expecting the "release-1.2" fork? I am sure you have
> > > > considered merging the FLIP-6 branch after the fork.
> > > >
> > > > Do we anticipate the new tests pushing Flink over Travis CI's new 50
> > > minute
> > > > limit? This might be a good opportunity to rebalance the test ranges
> as
> > > the
> > > > most recent passing master build (
> > > > https://travis-ci.org/apache/flink/builds/180504091) shows up to a
> 10
> > > > minute difference in runtime.
> > > >
> > > > Greg
> > > >
> > > > On Fri, Dec 2, 2016 at 11:37 AM, Stephan Ewen <se...@apache.org>
> > wrote:
> > > >
> > > > > Hi all!
> > > > >
> > > > > I want to start a discussion about merging the FLIP-6 feature
> branch
> > > into
> > > > > the master branch.
> > > > >
> > > > > The feature branch implements the following:
> > > > > - A new RPC abstraction
> > > > > - A new High-Availability Services Abstraction
> > > > > - New JobManager / TaskManager / ResourceManager with dynamic
> slot
> > > > > allocation
> > > > > - Re-designed runners to instantiate them
> > > > > - A new MiniCluster
> > > > >
> > > > > The feature branch still needs quite a bit of work, but it does not
> > > > > interfere with the current state of the master branch. All new
> > > components
> > > > > are implemented as separate components with separate tests.
> > > > >
> > > > > The branch builds fully, all tests pass, and when setting up Flink
> by
> > > any
> > > > > of the means supported in the Master, the same code is used and
> none
> > of
> > > > the
> > > > > feature branch code is active.
> > > > >
> > > > >
> > > > > At the same time, it becomes harder for the feature branch to chase
> > the
> > > > > master branch.
> > > > > Also, the feature branch starts to contain cleanups that are
> valuable
> > > in
> > > > > the master branch as well, for example around reusable parts like
> the
> > > > "high
> > > > > availability services".
> > > > >
> > > > >
> > > > > *My suggestion is the following:*
> > > > >
> > > > > - Let's merge the feature branch in the near future, i.e., quite
> > > soon.
> > > > >
> > > > > - When we fork off the "release-1.2" branch, we delete the new
> > > > components
> > > > > form that branch. That way we avoid having seemingly dead code in
> the
> > > > > source release.
> > > > >
> > > > > - The feature branch classes are mostly in some subpackages, so
> it
> > > > should
> > > > > be quite straight forward to remove them.
> > > > >
> > > > >
> > > > > What do you think about this?
> > > > >
> > > > >
> > > > > Greetings,
> > > > > Stephan
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Merging the FLIP-6 feature branch into the Master branch
Posted by Stephan Ewen <se...@apache.org>.
I would be +1 to merge the FLIP-6 branch to the master branch after the 1.2
branch is forked off, if we manage to do that in a timely fashion.
Would actually be safer that way...
On Tue, Dec 6, 2016 at 8:49 PM, Robert Metzger <rm...@apache.org> wrote:
> I've reactivated the 1.2 release thread and the actual list of release
> blockers is quite short as it seems.
> We could actually manage to fork off a branch for the 1.2 release next week
> (this really depends on how fast we get the blockers down)
> Would it be okay for the people working on "flip-6" to wait until next week
> to make the final decision here?
>
> On Fri, Dec 2, 2016 at 7:18 PM, Stephan Ewen <se...@apache.org> wrote:
>
> > Hi Greg!
> >
> > Yes, originally I was arguing to merge Flip-6 after forking off the 1.2
> > branch. That would be the preferable solution, but since it seems that
> the
> > 1.2 branch may still take a few weeks, I was thinking that we may want to
> > do that earlier.
> >
> > Many flip-6 contributors would like to be active around the Christmas
> time,
> > and I almost expect the 1.2 branch will not be forked off by then.
> >
> > Stephan
> >
> >
> > On Fri, Dec 2, 2016 at 6:16 PM, Greg Hogan <co...@greghogan.com> wrote:
> >
> > > Hi Stephan,
> > >
> > > How soon are you expecting the "release-1.2" fork? I am sure you have
> > > considered merging the FLIP-6 branch after the fork.
> > >
> > > Do we anticipate the new tests pushing Flink over Travis CI's new 50
> > minute
> > > limit? This might be a good opportunity to rebalance the test ranges as
> > the
> > > most recent passing master build (
> > > https://travis-ci.org/apache/flink/builds/180504091) shows up to a 10
> > > minute difference in runtime.
> > >
> > > Greg
> > >
> > > On Fri, Dec 2, 2016 at 11:37 AM, Stephan Ewen <se...@apache.org>
> wrote:
> > >
> > > > Hi all!
> > > >
> > > > I want to start a discussion about merging the FLIP-6 feature branch
> > into
> > > > the master branch.
> > > >
> > > > The feature branch implements the following:
> > > > - A new RPC abstraction
> > > > - A new High-Availability Services Abstraction
> > > > - New JobManager / TaskManager / ResourceManager with dynamic slot
> > > > allocation
> > > > - Re-designed runners to instantiate them
> > > > - A new MiniCluster
> > > >
> > > > The feature branch still needs quite a bit of work, but it does not
> > > > interfere with the current state of the master branch. All new
> > components
> > > > are implemented as separate components with separate tests.
> > > >
> > > > The branch builds fully, all tests pass, and when setting up Flink by
> > any
> > > > of the means supported in the Master, the same code is used and none
> of
> > > the
> > > > feature branch code is active.
> > > >
> > > >
> > > > At the same time, it becomes harder for the feature branch to chase
> the
> > > > master branch.
> > > > Also, the feature branch starts to contain cleanups that are valuable
> > in
> > > > the master branch as well, for example around reusable parts like the
> > > "high
> > > > availability services".
> > > >
> > > >
> > > > *My suggestion is the following:*
> > > >
> > > > - Let's merge the feature branch in the near future, i.e., quite
> > soon.
> > > >
> > > > - When we fork off the "release-1.2" branch, we delete the new
> > > components
> > > > form that branch. That way we avoid having seemingly dead code in the
> > > > source release.
> > > >
> > > > - The feature branch classes are mostly in some subpackages, so it
> > > should
> > > > be quite straight forward to remove them.
> > > >
> > > >
> > > > What do you think about this?
> > > >
> > > >
> > > > Greetings,
> > > > Stephan
> > > >
> > >
> >
>
Re: [DISCUSS] Merging the FLIP-6 feature branch into the Master branch
Posted by Robert Metzger <rm...@apache.org>.
I've reactivated the 1.2 release thread and the actual list of release
blockers is quite short as it seems.
We could actually manage to fork off a branch for the 1.2 release next week
(this really depends on how fast we get the blockers down)
Would it be okay for the people working on "flip-6" to wait until next week
to make the final decision here?
On Fri, Dec 2, 2016 at 7:18 PM, Stephan Ewen <se...@apache.org> wrote:
> Hi Greg!
>
> Yes, originally I was arguing to merge Flip-6 after forking off the 1.2
> branch. That would be the preferable solution, but since it seems that the
> 1.2 branch may still take a few weeks, I was thinking that we may want to
> do that earlier.
>
> Many flip-6 contributors would like to be active around the Christmas time,
> and I almost expect the 1.2 branch will not be forked off by then.
>
> Stephan
>
>
> On Fri, Dec 2, 2016 at 6:16 PM, Greg Hogan <co...@greghogan.com> wrote:
>
> > Hi Stephan,
> >
> > How soon are you expecting the "release-1.2" fork? I am sure you have
> > considered merging the FLIP-6 branch after the fork.
> >
> > Do we anticipate the new tests pushing Flink over Travis CI's new 50
> minute
> > limit? This might be a good opportunity to rebalance the test ranges as
> the
> > most recent passing master build (
> > https://travis-ci.org/apache/flink/builds/180504091) shows up to a 10
> > minute difference in runtime.
> >
> > Greg
> >
> > On Fri, Dec 2, 2016 at 11:37 AM, Stephan Ewen <se...@apache.org> wrote:
> >
> > > Hi all!
> > >
> > > I want to start a discussion about merging the FLIP-6 feature branch
> into
> > > the master branch.
> > >
> > > The feature branch implements the following:
> > > - A new RPC abstraction
> > > - A new High-Availability Services Abstraction
> > > - New JobManager / TaskManager / ResourceManager with dynamic slot
> > > allocation
> > > - Re-designed runners to instantiate them
> > > - A new MiniCluster
> > >
> > > The feature branch still needs quite a bit of work, but it does not
> > > interfere with the current state of the master branch. All new
> components
> > > are implemented as separate components with separate tests.
> > >
> > > The branch builds fully, all tests pass, and when setting up Flink by
> any
> > > of the means supported in the Master, the same code is used and none of
> > the
> > > feature branch code is active.
> > >
> > >
> > > At the same time, it becomes harder for the feature branch to chase the
> > > master branch.
> > > Also, the feature branch starts to contain cleanups that are valuable
> in
> > > the master branch as well, for example around reusable parts like the
> > "high
> > > availability services".
> > >
> > >
> > > *My suggestion is the following:*
> > >
> > > - Let's merge the feature branch in the near future, i.e., quite
> soon.
> > >
> > > - When we fork off the "release-1.2" branch, we delete the new
> > components
> > > form that branch. That way we avoid having seemingly dead code in the
> > > source release.
> > >
> > > - The feature branch classes are mostly in some subpackages, so it
> > should
> > > be quite straight forward to remove them.
> > >
> > >
> > > What do you think about this?
> > >
> > >
> > > Greetings,
> > > Stephan
> > >
> >
>
Re: [DISCUSS] Merging the FLIP-6 feature branch into the Master branch
Posted by Stephan Ewen <se...@apache.org>.
Hi Greg!
Yes, originally I was arguing to merge Flip-6 after forking off the 1.2
branch. That would be the preferable solution, but since it seems that the
1.2 branch may still take a few weeks, I was thinking that we may want to
do that earlier.
Many flip-6 contributors would like to be active around the Christmas time,
and I almost expect the 1.2 branch will not be forked off by then.
Stephan
On Fri, Dec 2, 2016 at 6:16 PM, Greg Hogan <co...@greghogan.com> wrote:
> Hi Stephan,
>
> How soon are you expecting the "release-1.2" fork? I am sure you have
> considered merging the FLIP-6 branch after the fork.
>
> Do we anticipate the new tests pushing Flink over Travis CI's new 50 minute
> limit? This might be a good opportunity to rebalance the test ranges as the
> most recent passing master build (
> https://travis-ci.org/apache/flink/builds/180504091) shows up to a 10
> minute difference in runtime.
>
> Greg
>
> On Fri, Dec 2, 2016 at 11:37 AM, Stephan Ewen <se...@apache.org> wrote:
>
> > Hi all!
> >
> > I want to start a discussion about merging the FLIP-6 feature branch into
> > the master branch.
> >
> > The feature branch implements the following:
> > - A new RPC abstraction
> > - A new High-Availability Services Abstraction
> > - New JobManager / TaskManager / ResourceManager with dynamic slot
> > allocation
> > - Re-designed runners to instantiate them
> > - A new MiniCluster
> >
> > The feature branch still needs quite a bit of work, but it does not
> > interfere with the current state of the master branch. All new components
> > are implemented as separate components with separate tests.
> >
> > The branch builds fully, all tests pass, and when setting up Flink by any
> > of the means supported in the Master, the same code is used and none of
> the
> > feature branch code is active.
> >
> >
> > At the same time, it becomes harder for the feature branch to chase the
> > master branch.
> > Also, the feature branch starts to contain cleanups that are valuable in
> > the master branch as well, for example around reusable parts like the
> "high
> > availability services".
> >
> >
> > *My suggestion is the following:*
> >
> > - Let's merge the feature branch in the near future, i.e., quite soon.
> >
> > - When we fork off the "release-1.2" branch, we delete the new
> components
> > form that branch. That way we avoid having seemingly dead code in the
> > source release.
> >
> > - The feature branch classes are mostly in some subpackages, so it
> should
> > be quite straight forward to remove them.
> >
> >
> > What do you think about this?
> >
> >
> > Greetings,
> > Stephan
> >
>
Re: [DISCUSS] Merging the FLIP-6 feature branch into the Master branch
Posted by Greg Hogan <co...@greghogan.com>.
Hi Stephan,
How soon are you expecting the "release-1.2" fork? I am sure you have
considered merging the FLIP-6 branch after the fork.
Do we anticipate the new tests pushing Flink over Travis CI's new 50 minute
limit? This might be a good opportunity to rebalance the test ranges as the
most recent passing master build (
https://travis-ci.org/apache/flink/builds/180504091) shows up to a 10
minute difference in runtime.
Greg
On Fri, Dec 2, 2016 at 11:37 AM, Stephan Ewen <se...@apache.org> wrote:
> Hi all!
>
> I want to start a discussion about merging the FLIP-6 feature branch into
> the master branch.
>
> The feature branch implements the following:
> - A new RPC abstraction
> - A new High-Availability Services Abstraction
> - New JobManager / TaskManager / ResourceManager with dynamic slot
> allocation
> - Re-designed runners to instantiate them
> - A new MiniCluster
>
> The feature branch still needs quite a bit of work, but it does not
> interfere with the current state of the master branch. All new components
> are implemented as separate components with separate tests.
>
> The branch builds fully, all tests pass, and when setting up Flink by any
> of the means supported in the Master, the same code is used and none of the
> feature branch code is active.
>
>
> At the same time, it becomes harder for the feature branch to chase the
> master branch.
> Also, the feature branch starts to contain cleanups that are valuable in
> the master branch as well, for example around reusable parts like the "high
> availability services".
>
>
> *My suggestion is the following:*
>
> - Let's merge the feature branch in the near future, i.e., quite soon.
>
> - When we fork off the "release-1.2" branch, we delete the new components
> form that branch. That way we avoid having seemingly dead code in the
> source release.
>
> - The feature branch classes are mostly in some subpackages, so it should
> be quite straight forward to remove them.
>
>
> What do you think about this?
>
>
> Greetings,
> Stephan
>