You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Konstantin Knauf <kn...@apache.org> on 2022/02/01 09:27:17 UTC
Re: [VOTE] Deprecate Per-Job Mode in Flink 1.15
Hi Xintong, Hi Yang, Hi everyone,
Thank you for speaking up. The vote is formally only about the deprecation
in Flink 1.15.
We can and should continue to collect blockers for the deletion of per-job
mode on YARN. Then there should be one release that allows users to switch.
So, Flink 1.16 indeed is unrealistic for dropping, as we would need to
address all Blockers still in Flink 1.15.
I think a certain degree of urgency helps us to address these issues and
encourages users to switch to application mode. So, I would continue to
target Flink 1.17 for dropping per-job mode, but let's reevaluate after
Flink 1.16.
Hope this helps,
Konstantin
Since we recently decided that
On Sun, Jan 30, 2022 at 4:13 AM Yang Wang <da...@gmail.com> wrote:
> Hi all,
>
>
>
> I second Xintong’s comments to not drop the per-job mode too aggressively.
> And I am afraid
>
> we need to get more inputs from users after deprecating the per-job mode in
> release-1.15.
>
>
> Most Flink on YARN users are using CLI command to integrate with the job
> lifecycle management system.
>
> And they are still using the old compatibility mode "flink run -m
> yarn-cluster", not the generic CLI mode "--target
> yarn-per-job/yarn-application".
>
> Apart from the functionalities, they need some time to upgrade the external
> systems.
>
>
> BTW, the application mode does not support attached mode now. Some users
> have asked for this in FLINK-25495[1].
>
>
> [1]. https://issues.apache.org/jira/browse/FLINK-25495
>
>
>
>
> Best,
>
> Yang
>
> Xintong Song <to...@gmail.com> 于2022年1月30日周日 08:35写道：
>
> > Hi Konstantin,
> >
> > Could we be more specific about what this vote is for? I'm asking
> because I
> > don't think we have consensus on all you have mentioned.
> >
> > To be specific, I'd be +1 for deprecating per-job mode in 1.15. However,
> > I'm not sure about the following.
> > - Targeting to drop it in 1.16 or 1.17. TBH, I'd expect to stay
> compatible
> > on the per-job mode a bit longer.
> > - Targeting Yarn application mode on par with the standalone / K8s. I
> think
> > we need the Yarn application mode on par with the Yarn per-job mode, as
> the
> > latter is being dropped and users are migrating from.
> > - FLINK-24897 being the only blocker for dropping the per-job mode. I
> think
> > a good time to drop the per-job mode is probably when we know most users
> > have migrated to the application mode. Even if the Yarn application mode
> > provides equivalent functionality as the Yarn per-job mode does, it's
> > probably nicer to not force users to migrate if the per-job mode is still
> > widely used.
> >
> > Discussing the above items is not my purpose here. Just trying to say
> that
> > IMHO in the previous discussion [1] we have not reached consensus on all
> > the things mentioned in this voting thread. Consequently, if these are
> all
> > included in the scope of the vote, I'm afraid I cannot give my +1 on
> this.
> > Sorry if I'm nitpicking.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> > [1] https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n
> >
> > On Sat, Jan 29, 2022 at 2:27 PM Jing Zhang <be...@gmail.com> wrote:
> >
> > > +1 (binding)
> > >
> > > Thanks Konstantin for driving this.
> > >
> > > Best,
> > > Jing Zhang
> > >
> > > Chenya Zhang <ch...@gmail.com> 于2022年1月29日周六 07:04写道：
> > >
> > > > +1 (non-binding)
> > > >
> > > > On Fri, Jan 28, 2022 at 12:46 PM Thomas Weise <th...@apache.org>
> wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > On Fri, Jan 28, 2022 at 9:27 AM David Morávek <dm...@apache.org>
> > wrote:
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Fri 28. 1. 2022 at 17:53, Till Rohrmann <trohrmann@apache.org
> >
> > > > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Till
> > > > > > >
> > > > > > > On Fri, Jan 28, 2022 at 4:57 PM Gabor Somogyi <
> > > > > gabor.g.somogyi@gmail.com
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > +1 (non-binding)
> > > > > > > >
> > > > > > > > We're intended to make tests when FLINK-24897
> > > > > > > > <https://issues.apache.org/jira/browse/FLINK-24897> is
> fixed.
> > > > > > > > In case of further issues we're going to create further
> jiras.
> > > > > > > >
> > > > > > > > BR,
> > > > > > > > G
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Jan 28, 2022 at 4:30 PM Konstantin Knauf <
> > > > knaufk@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi everyone,
> > > > > > > > >
> > > > > > > > > Based on the discussion in [1], I would like to start a
> vote
> > on
> > > > > > > > deprecating
> > > > > > > > > per-job mode in Flink 1.15. Consequently, we would target
> to
> > > drop
> > > > > it
> > > > > > in
> > > > > > > > > Flink 1.16 or Flink 1.17 latest.
> > > > > > > > >
> > > > > > > > > The only limitation that would block dropping Per-Job mode
> > > > > mentioned
> > > > > > in
> > > > > > > > [1]
> > > > > > > > > is tracked in
> > > https://issues.apache.org/jira/browse/FLINK-24897.
> > > > > In
> > > > > > > > > general, the implementation of application mode in YARN
> > should
> > > be
> > > > > on
> > > > > > > par
> > > > > > > > > with the standalone and Kubernetes before we drop per-job
> > mode.
> > > > > > > > >
> > > > > > > > > The vote will last for at least 72 hours, and will be
> > accepted
> > > > by a
> > > > > > > > > consensus of active committers.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Konstantin
> > > > > > > > >
> > > > > > > > > [1]
> > > > > https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > Konstantin Knauf
> > > > > > > > >
> > > > > > > > > https://twitter.com/snntrable
> > > > > > > > >
> > > > > > > > > https://github.com/knaufk
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--
Konstantin Knauf
https://twitter.com/snntrable
https://github.com/knaufk
Re: [VOTE] Deprecate Per-Job Mode in Flink 1.15
Posted by Konstantin Knauf <kn...@apache.org>.
Thank you everyone, I've closed the vote and created a ticket for
deprecation [1] and dropping [2] and linked the current blockers for
dropping it to the latter.
Please if or when you encounter new blockers link them to [2].
[1] https://issues.apache.org/jira/browse/FLINK-25999
[2] https://issues.apache.org/jira/browse/FLINK-26000
On Sun, Feb 6, 2022 at 6:44 AM Yang Wang <da...@gmail.com> wrote:
> Thanks Konstantin for the explanation.
>
> +1 (binding) from me now.
>
>
> Best,
> Yang
>
> Xintong Song <to...@gmail.com> 于2022年2月3日周四 09:42写道：
>
> > Thanks for the clarification, Konstantin.
> >
> > +1 for deprecating per-job mode in Flink 1.15, and reevaluating when to
> > drop it after Flink 1.16.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Tue, Feb 1, 2022 at 5:27 PM Konstantin Knauf <kn...@apache.org>
> wrote:
> >
> > > Hi Xintong, Hi Yang, Hi everyone,
> > >
> > > Thank you for speaking up. The vote is formally only about the
> > deprecation
> > > in Flink 1.15.
> > >
> > > We can and should continue to collect blockers for the deletion of
> > per-job
> > > mode on YARN. Then there should be one release that allows users to
> > switch.
> > > So, Flink 1.16 indeed is unrealistic for dropping, as we would need to
> > > address all Blockers still in Flink 1.15.
> > >
> > > I think a certain degree of urgency helps us to address these issues
> and
> > > encourages users to switch to application mode. So, I would continue to
> > > target Flink 1.17 for dropping per-job mode, but let's reevaluate after
> > > Flink 1.16.
> > >
> > > Hope this helps,
> > >
> > > Konstantin
> > >
> > > Since we recently decided that
> > > On Sun, Jan 30, 2022 at 4:13 AM Yang Wang <da...@gmail.com>
> wrote:
> > >
> > > > Hi all,
> > > >
> > > >
> > > >
> > > > I second Xintong’s comments to not drop the per-job mode too
> > > aggressively.
> > > > And I am afraid
> > > >
> > > > we need to get more inputs from users after deprecating the per-job
> > mode
> > > in
> > > > release-1.15.
> > > >
> > > >
> > > > Most Flink on YARN users are using CLI command to integrate with the
> > job
> > > > lifecycle management system.
> > > >
> > > > And they are still using the old compatibility mode "flink run -m
> > > > yarn-cluster", not the generic CLI mode "--target
> > > > yarn-per-job/yarn-application".
> > > >
> > > > Apart from the functionalities, they need some time to upgrade the
> > > external
> > > > systems.
> > > >
> > > >
> > > > BTW, the application mode does not support attached mode now. Some
> > users
> > > > have asked for this in FLINK-25495[1].
> > > >
> > > >
> > > > [1]. https://issues.apache.org/jira/browse/FLINK-25495
> > > >
> > > >
> > > >
> > > >
> > > > Best,
> > > >
> > > > Yang
> > > >
> > > > Xintong Song <to...@gmail.com> 于2022年1月30日周日 08:35写道：
> > > >
> > > > > Hi Konstantin,
> > > > >
> > > > > Could we be more specific about what this vote is for? I'm asking
> > > > because I
> > > > > don't think we have consensus on all you have mentioned.
> > > > >
> > > > > To be specific, I'd be +1 for deprecating per-job mode in 1.15.
> > > However,
> > > > > I'm not sure about the following.
> > > > > - Targeting to drop it in 1.16 or 1.17. TBH, I'd expect to stay
> > > > compatible
> > > > > on the per-job mode a bit longer.
> > > > > - Targeting Yarn application mode on par with the standalone /
> K8s. I
> > > > think
> > > > > we need the Yarn application mode on par with the Yarn per-job
> mode,
> > as
> > > > the
> > > > > latter is being dropped and users are migrating from.
> > > > > - FLINK-24897 being the only blocker for dropping the per-job
> mode. I
> > > > think
> > > > > a good time to drop the per-job mode is probably when we know most
> > > users
> > > > > have migrated to the application mode. Even if the Yarn application
> > > mode
> > > > > provides equivalent functionality as the Yarn per-job mode does,
> it's
> > > > > probably nicer to not force users to migrate if the per-job mode is
> > > still
> > > > > widely used.
> > > > >
> > > > > Discussing the above items is not my purpose here. Just trying to
> say
> > > > that
> > > > > IMHO in the previous discussion [1] we have not reached consensus
> on
> > > all
> > > > > the things mentioned in this voting thread. Consequently, if these
> > are
> > > > all
> > > > > included in the scope of the vote, I'm afraid I cannot give my +1
> on
> > > > this.
> > > > > Sorry if I'm nitpicking.
> > > > >
> > > > > Thank you~
> > > > >
> > > > > Xintong Song
> > > > >
> > > > >
> > > > > [1]
> https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n
> > > > >
> > > > > On Sat, Jan 29, 2022 at 2:27 PM Jing Zhang <be...@gmail.com>
> > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Thanks Konstantin for driving this.
> > > > > >
> > > > > > Best,
> > > > > > Jing Zhang
> > > > > >
> > > > > > Chenya Zhang <ch...@gmail.com> 于2022年1月29日周六
> 07:04写道：
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > On Fri, Jan 28, 2022 at 12:46 PM Thomas Weise <th...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > On Fri, Jan 28, 2022 at 9:27 AM David Morávek <
> dmvk@apache.org
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 (non-binding)
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Fri 28. 1. 2022 at 17:53, Till Rohrmann <
> > > trohrmann@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > +1 (binding)
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > > Till
> > > > > > > > > >
> > > > > > > > > > On Fri, Jan 28, 2022 at 4:57 PM Gabor Somogyi <
> > > > > > > > gabor.g.somogyi@gmail.com
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > +1 (non-binding)
> > > > > > > > > > >
> > > > > > > > > > > We're intended to make tests when FLINK-24897
> > > > > > > > > > > <https://issues.apache.org/jira/browse/FLINK-24897> is
> > > > fixed.
> > > > > > > > > > > In case of further issues we're going to create further
> > > > jiras.
> > > > > > > > > > >
> > > > > > > > > > > BR,
> > > > > > > > > > > G
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Jan 28, 2022 at 4:30 PM Konstantin Knauf <
> > > > > > > knaufk@apache.org>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi everyone,
> > > > > > > > > > > >
> > > > > > > > > > > > Based on the discussion in [1], I would like to
> start a
> > > > vote
> > > > > on
> > > > > > > > > > > deprecating
> > > > > > > > > > > > per-job mode in Flink 1.15. Consequently, we would
> > target
> > > > to
> > > > > > drop
> > > > > > > > it
> > > > > > > > > in
> > > > > > > > > > > > Flink 1.16 or Flink 1.17 latest.
> > > > > > > > > > > >
> > > > > > > > > > > > The only limitation that would block dropping Per-Job
> > > mode
> > > > > > > > mentioned
> > > > > > > > > in
> > > > > > > > > > > [1]
> > > > > > > > > > > > is tracked in
> > > > > > https://issues.apache.org/jira/browse/FLINK-24897.
> > > > > > > > In
> > > > > > > > > > > > general, the implementation of application mode in
> YARN
> > > > > should
> > > > > > be
> > > > > > > > on
> > > > > > > > > > par
> > > > > > > > > > > > with the standalone and Kubernetes before we drop
> > per-job
> > > > > mode.
> > > > > > > > > > > >
> > > > > > > > > > > > The vote will last for at least 72 hours, and will be
> > > > > accepted
> > > > > > > by a
> > > > > > > > > > > > consensus of active committers.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > >
> > > > > > > > > > > > Konstantin
> > > > > > > > > > > >
> > > > > > > > > > > > [1]
> > > > > > > >
> > https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > >
> > > > > > > > > > > > Konstantin Knauf
> > > > > > > > > > > >
> > > > > > > > > > > > https://twitter.com/snntrable
> > > > > > > > > > > >
> > > > > > > > > > > > https://github.com/knaufk
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Konstantin Knauf
> > >
> > > https://twitter.com/snntrable
> > >
> > > https://github.com/knaufk
> > >
> >
>
--
Konstantin Knauf
https://twitter.com/snntrable
https://github.com/knaufk
Re: [VOTE] Deprecate Per-Job Mode in Flink 1.15
Posted by Yang Wang <da...@gmail.com>.
Thanks Konstantin for the explanation.
+1 (binding) from me now.
Best,
Yang
Xintong Song <to...@gmail.com> 于2022年2月3日周四 09:42写道：
> Thanks for the clarification, Konstantin.
>
> +1 for deprecating per-job mode in Flink 1.15, and reevaluating when to
> drop it after Flink 1.16.
>
> Thank you~
>
> Xintong Song
>
>
>
> On Tue, Feb 1, 2022 at 5:27 PM Konstantin Knauf <kn...@apache.org> wrote:
>
> > Hi Xintong, Hi Yang, Hi everyone,
> >
> > Thank you for speaking up. The vote is formally only about the
> deprecation
> > in Flink 1.15.
> >
> > We can and should continue to collect blockers for the deletion of
> per-job
> > mode on YARN. Then there should be one release that allows users to
> switch.
> > So, Flink 1.16 indeed is unrealistic for dropping, as we would need to
> > address all Blockers still in Flink 1.15.
> >
> > I think a certain degree of urgency helps us to address these issues and
> > encourages users to switch to application mode. So, I would continue to
> > target Flink 1.17 for dropping per-job mode, but let's reevaluate after
> > Flink 1.16.
> >
> > Hope this helps,
> >
> > Konstantin
> >
> > Since we recently decided that
> > On Sun, Jan 30, 2022 at 4:13 AM Yang Wang <da...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > >
> > >
> > > I second Xintong’s comments to not drop the per-job mode too
> > aggressively.
> > > And I am afraid
> > >
> > > we need to get more inputs from users after deprecating the per-job
> mode
> > in
> > > release-1.15.
> > >
> > >
> > > Most Flink on YARN users are using CLI command to integrate with the
> job
> > > lifecycle management system.
> > >
> > > And they are still using the old compatibility mode "flink run -m
> > > yarn-cluster", not the generic CLI mode "--target
> > > yarn-per-job/yarn-application".
> > >
> > > Apart from the functionalities, they need some time to upgrade the
> > external
> > > systems.
> > >
> > >
> > > BTW, the application mode does not support attached mode now. Some
> users
> > > have asked for this in FLINK-25495[1].
> > >
> > >
> > > [1]. https://issues.apache.org/jira/browse/FLINK-25495
> > >
> > >
> > >
> > >
> > > Best,
> > >
> > > Yang
> > >
> > > Xintong Song <to...@gmail.com> 于2022年1月30日周日 08:35写道：
> > >
> > > > Hi Konstantin,
> > > >
> > > > Could we be more specific about what this vote is for? I'm asking
> > > because I
> > > > don't think we have consensus on all you have mentioned.
> > > >
> > > > To be specific, I'd be +1 for deprecating per-job mode in 1.15.
> > However,
> > > > I'm not sure about the following.
> > > > - Targeting to drop it in 1.16 or 1.17. TBH, I'd expect to stay
> > > compatible
> > > > on the per-job mode a bit longer.
> > > > - Targeting Yarn application mode on par with the standalone / K8s. I
> > > think
> > > > we need the Yarn application mode on par with the Yarn per-job mode,
> as
> > > the
> > > > latter is being dropped and users are migrating from.
> > > > - FLINK-24897 being the only blocker for dropping the per-job mode. I
> > > think
> > > > a good time to drop the per-job mode is probably when we know most
> > users
> > > > have migrated to the application mode. Even if the Yarn application
> > mode
> > > > provides equivalent functionality as the Yarn per-job mode does, it's
> > > > probably nicer to not force users to migrate if the per-job mode is
> > still
> > > > widely used.
> > > >
> > > > Discussing the above items is not my purpose here. Just trying to say
> > > that
> > > > IMHO in the previous discussion [1] we have not reached consensus on
> > all
> > > > the things mentioned in this voting thread. Consequently, if these
> are
> > > all
> > > > included in the scope of the vote, I'm afraid I cannot give my +1 on
> > > this.
> > > > Sorry if I'm nitpicking.
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song
> > > >
> > > >
> > > > [1] https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n
> > > >
> > > > On Sat, Jan 29, 2022 at 2:27 PM Jing Zhang <be...@gmail.com>
> > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks Konstantin for driving this.
> > > > >
> > > > > Best,
> > > > > Jing Zhang
> > > > >
> > > > > Chenya Zhang <ch...@gmail.com> 于2022年1月29日周六 07:04写道：
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > On Fri, Jan 28, 2022 at 12:46 PM Thomas Weise <th...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > On Fri, Jan 28, 2022 at 9:27 AM David Morávek <dmvk@apache.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > > +1 (non-binding)
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Fri 28. 1. 2022 at 17:53, Till Rohrmann <
> > trohrmann@apache.org
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 (binding)
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Till
> > > > > > > > >
> > > > > > > > > On Fri, Jan 28, 2022 at 4:57 PM Gabor Somogyi <
> > > > > > > gabor.g.somogyi@gmail.com
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > +1 (non-binding)
> > > > > > > > > >
> > > > > > > > > > We're intended to make tests when FLINK-24897
> > > > > > > > > > <https://issues.apache.org/jira/browse/FLINK-24897> is
> > > fixed.
> > > > > > > > > > In case of further issues we're going to create further
> > > jiras.
> > > > > > > > > >
> > > > > > > > > > BR,
> > > > > > > > > > G
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Jan 28, 2022 at 4:30 PM Konstantin Knauf <
> > > > > > knaufk@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi everyone,
> > > > > > > > > > >
> > > > > > > > > > > Based on the discussion in [1], I would like to start a
> > > vote
> > > > on
> > > > > > > > > > deprecating
> > > > > > > > > > > per-job mode in Flink 1.15. Consequently, we would
> target
> > > to
> > > > > drop
> > > > > > > it
> > > > > > > > in
> > > > > > > > > > > Flink 1.16 or Flink 1.17 latest.
> > > > > > > > > > >
> > > > > > > > > > > The only limitation that would block dropping Per-Job
> > mode
> > > > > > > mentioned
> > > > > > > > in
> > > > > > > > > > [1]
> > > > > > > > > > > is tracked in
> > > > > https://issues.apache.org/jira/browse/FLINK-24897.
> > > > > > > In
> > > > > > > > > > > general, the implementation of application mode in YARN
> > > > should
> > > > > be
> > > > > > > on
> > > > > > > > > par
> > > > > > > > > > > with the standalone and Kubernetes before we drop
> per-job
> > > > mode.
> > > > > > > > > > >
> > > > > > > > > > > The vote will last for at least 72 hours, and will be
> > > > accepted
> > > > > > by a
> > > > > > > > > > > consensus of active committers.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > >
> > > > > > > > > > > Konstantin
> > > > > > > > > > >
> > > > > > > > > > > [1]
> > > > > > >
> https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > >
> > > > > > > > > > > Konstantin Knauf
> > > > > > > > > > >
> > > > > > > > > > > https://twitter.com/snntrable
> > > > > > > > > > >
> > > > > > > > > > > https://github.com/knaufk
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Konstantin Knauf
> >
> > https://twitter.com/snntrable
> >
> > https://github.com/knaufk
> >
>
Re: [VOTE] Deprecate Per-Job Mode in Flink 1.15
Posted by Xintong Song <to...@gmail.com>.
Thanks for the clarification, Konstantin.
+1 for deprecating per-job mode in Flink 1.15, and reevaluating when to
drop it after Flink 1.16.
Thank you~
Xintong Song
On Tue, Feb 1, 2022 at 5:27 PM Konstantin Knauf <kn...@apache.org> wrote:
> Hi Xintong, Hi Yang, Hi everyone,
>
> Thank you for speaking up. The vote is formally only about the deprecation
> in Flink 1.15.
>
> We can and should continue to collect blockers for the deletion of per-job
> mode on YARN. Then there should be one release that allows users to switch.
> So, Flink 1.16 indeed is unrealistic for dropping, as we would need to
> address all Blockers still in Flink 1.15.
>
> I think a certain degree of urgency helps us to address these issues and
> encourages users to switch to application mode. So, I would continue to
> target Flink 1.17 for dropping per-job mode, but let's reevaluate after
> Flink 1.16.
>
> Hope this helps,
>
> Konstantin
>
> Since we recently decided that
> On Sun, Jan 30, 2022 at 4:13 AM Yang Wang <da...@gmail.com> wrote:
>
> > Hi all,
> >
> >
> >
> > I second Xintong’s comments to not drop the per-job mode too
> aggressively.
> > And I am afraid
> >
> > we need to get more inputs from users after deprecating the per-job mode
> in
> > release-1.15.
> >
> >
> > Most Flink on YARN users are using CLI command to integrate with the job
> > lifecycle management system.
> >
> > And they are still using the old compatibility mode "flink run -m
> > yarn-cluster", not the generic CLI mode "--target
> > yarn-per-job/yarn-application".
> >
> > Apart from the functionalities, they need some time to upgrade the
> external
> > systems.
> >
> >
> > BTW, the application mode does not support attached mode now. Some users
> > have asked for this in FLINK-25495[1].
> >
> >
> > [1]. https://issues.apache.org/jira/browse/FLINK-25495
> >
> >
> >
> >
> > Best,
> >
> > Yang
> >
> > Xintong Song <to...@gmail.com> 于2022年1月30日周日 08:35写道：
> >
> > > Hi Konstantin,
> > >
> > > Could we be more specific about what this vote is for? I'm asking
> > because I
> > > don't think we have consensus on all you have mentioned.
> > >
> > > To be specific, I'd be +1 for deprecating per-job mode in 1.15.
> However,
> > > I'm not sure about the following.
> > > - Targeting to drop it in 1.16 or 1.17. TBH, I'd expect to stay
> > compatible
> > > on the per-job mode a bit longer.
> > > - Targeting Yarn application mode on par with the standalone / K8s. I
> > think
> > > we need the Yarn application mode on par with the Yarn per-job mode, as
> > the
> > > latter is being dropped and users are migrating from.
> > > - FLINK-24897 being the only blocker for dropping the per-job mode. I
> > think
> > > a good time to drop the per-job mode is probably when we know most
> users
> > > have migrated to the application mode. Even if the Yarn application
> mode
> > > provides equivalent functionality as the Yarn per-job mode does, it's
> > > probably nicer to not force users to migrate if the per-job mode is
> still
> > > widely used.
> > >
> > > Discussing the above items is not my purpose here. Just trying to say
> > that
> > > IMHO in the previous discussion [1] we have not reached consensus on
> all
> > > the things mentioned in this voting thread. Consequently, if these are
> > all
> > > included in the scope of the vote, I'm afraid I cannot give my +1 on
> > this.
> > > Sorry if I'm nitpicking.
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > > [1] https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n
> > >
> > > On Sat, Jan 29, 2022 at 2:27 PM Jing Zhang <be...@gmail.com>
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Thanks Konstantin for driving this.
> > > >
> > > > Best,
> > > > Jing Zhang
> > > >
> > > > Chenya Zhang <ch...@gmail.com> 于2022年1月29日周六 07:04写道：
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > On Fri, Jan 28, 2022 at 12:46 PM Thomas Weise <th...@apache.org>
> > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > On Fri, Jan 28, 2022 at 9:27 AM David Morávek <dm...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Fri 28. 1. 2022 at 17:53, Till Rohrmann <
> trohrmann@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > > Till
> > > > > > > >
> > > > > > > > On Fri, Jan 28, 2022 at 4:57 PM Gabor Somogyi <
> > > > > > gabor.g.somogyi@gmail.com
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 (non-binding)
> > > > > > > > >
> > > > > > > > > We're intended to make tests when FLINK-24897
> > > > > > > > > <https://issues.apache.org/jira/browse/FLINK-24897> is
> > fixed.
> > > > > > > > > In case of further issues we're going to create further
> > jiras.
> > > > > > > > >
> > > > > > > > > BR,
> > > > > > > > > G
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Jan 28, 2022 at 4:30 PM Konstantin Knauf <
> > > > > knaufk@apache.org>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi everyone,
> > > > > > > > > >
> > > > > > > > > > Based on the discussion in [1], I would like to start a
> > vote
> > > on
> > > > > > > > > deprecating
> > > > > > > > > > per-job mode in Flink 1.15. Consequently, we would target
> > to
> > > > drop
> > > > > > it
> > > > > > > in
> > > > > > > > > > Flink 1.16 or Flink 1.17 latest.
> > > > > > > > > >
> > > > > > > > > > The only limitation that would block dropping Per-Job
> mode
> > > > > > mentioned
> > > > > > > in
> > > > > > > > > [1]
> > > > > > > > > > is tracked in
> > > > https://issues.apache.org/jira/browse/FLINK-24897.
> > > > > > In
> > > > > > > > > > general, the implementation of application mode in YARN
> > > should
> > > > be
> > > > > > on
> > > > > > > > par
> > > > > > > > > > with the standalone and Kubernetes before we drop per-job
> > > mode.
> > > > > > > > > >
> > > > > > > > > > The vote will last for at least 72 hours, and will be
> > > accepted
> > > > > by a
> > > > > > > > > > consensus of active committers.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > Konstantin
> > > > > > > > > >
> > > > > > > > > > [1]
> > > > > > https://lists.apache.org/thread/b8g76cqgtr2c515rd1bs41vy285f317n
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > > > Konstantin Knauf
> > > > > > > > > >
> > > > > > > > > > https://twitter.com/snntrable
> > > > > > > > > >
> > > > > > > > > > https://github.com/knaufk
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>