You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Kostas Kloudas <kk...@gmail.com> on 2019/08/14 09:08:50 UTC
Re: [DISCUSS] Drop stale class Program
Hi all,
It is nice to have this discussion.
I am totally up for removing the unused Program interface, as this will
simplify a lot of other code paths in the ClusterClient and elsewhere.
Also about the easier integration of Flink with other frameworks, there
is another discussion in the mailing list with exactly this topic:
[DISCUSS] Flink client api enhancement for downstream project
Cheers,
Kostas
On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <wa...@gmail.com> wrote:
> Hi,
>
> With a one-week survey in user list[1], nobody expect Flavio and Jeff
> participant the thread.
>
> Flavio shared his experience with a revised Program like interface.
> This could be regraded as downstream integration and in client api
> enhancements document we propose rich interface for this integration.
> Anyway, the Flink scope Program is less functional than it should be.
>
> With no objection I'd like to push on this thread. We need a committer
> participant this thread to shepherd the removal/deprecation of Program, a
> @PublicEvolving interface. Anybody has spare time? Or anything I can do
> to make progress?
>
> Best,
> tison.
>
> [1]
>
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>
>
> Zili Chen <wa...@gmail.com> 于2019年7月22日周一 下午8:38写道：
>
> > Hi,
> >
> > I created a thread for survey in user list[1]. Please take participate in
> > if interested.
> >
> > Best,
> > tison.
> >
> > [1]
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> >
> >
> > Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午5:18写道：
> >
> >> +1 to remove directly the Program class (I think nobody use it and it's
> >> not
> >> supported at all by REST services and UI).
> >> Moreover it requires a lot of transitive dependencies while it should
> be a
> >> very simple thing..
> >> +1 to add this discussion to "Flink client api enhancement"
> >>
> >> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <mm...@gmail.com> wrote:
> >>
> >> > To Flavio, good point for the integration suggestion.
> >> >
> >> > I think it should be considered in the "Flink client api enhancement"
> >> > discussion. But the outdated API should be deprecated somehow.
> >> >
> >> > Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午4:21写道：
> >> >
> >> > > In my experience a basic "official" (but optional) program
> description
> >> > > would be very useful indeed (in order to ease the integration with
> >> other
> >> > > frameworks).
> >> > >
> >> > > Of course it should be extended and integrated with the REST
> services
> >> and
> >> > > the Web UI (when defined) in order to be useful..
> >> > > It ease to show to the user what a job does and which parameters it
> >> > > requires (optional or mandatory) and with a proper help description.
> >> > > Indeed, when we write a Flink job we implement the following
> >> interface:
> >> > >
> >> > > public interface FlinkJob {
> >> > > String getDescription();
> >> > > List<FlinkJobParameter> getParameters();
> >> > > boolean isStreamingOrBatch();
> >> > > }
> >> > >
> >> > > public class ClusterJobParameter {
> >> > >
> >> > > private String paramName;
> >> > > private String paramType = "string";
> >> > > private String paramDesc;
> >> > > private String paramDefaultValue;
> >> > > private Set<String> choices;
> >> > > private boolean mandatory;
> >> > > }
> >> > >
> >> > > This really helps to launch a Flink job by a frontend (if the rest
> >> > services
> >> > > returns back those infos).
> >> > >
> >> > > Best,
> >> > > Flavio
> >> > >
> >> > > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <mm...@gmail.com>
> wrote:
> >> > >
> >> > > > Hi Zili,
> >> > > >
> >> > > > Thank you for bring us this discussion.
> >> > > >
> >> > > > My gut feeling is +1 for dropping it.
> >> > > > Usually it costs some time to deprecate a public (actually it's
> >> > > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> >> > first.
> >> > > > Then it might be abandoned it in some later version.
> >> > > >
> >> > > > I'm not sure how big the burden is to make it compatible with the
> >> > > enhanced
> >> > > > client API. If it's a critical blocker, I support dropping it
> >> radically
> >> > > in
> >> > > > 1.10. Of course a survey is necessary. And the result of survey is
> >> > > > acceptable.
> >> > > >
> >> > > >
> >> > > >
> >> > > > Zili Chen <wa...@gmail.com> 于2019年7月19日周五 下午1:44写道：
> >> > > >
> >> > > > > Hi devs,
> >> > > > >
> >> > > > > Participating the thread "Flink client api enhancement"[1], I
> just
> >> > > notice
> >> > > > > that inside submission codepath of Flink we always has a branch
> >> > dealing
> >> > > > > with the case that main class of user program is a subclass of
> >> > > > > o.a.f.api.common.Program, which is defined as
> >> > > > >
> >> > > > > @PublicEvolving
> >> > > > > public interface Program {
> >> > > > > Plan getPhan(String... args);
> >> > > > > }
> >> > > > >
> >> > > > > This class, as user-facing interface, asks user to implement
> >> #getPlan
> >> > > > > which return a almost Flink internal class. FLINK-10862[2]
> pointed
> >> > out
> >> > > > > this confusion.
> >> > > > >
> >> > > > > Since our codebase contains quite a bit code handling this stale
> >> > class,
> >> > > > > and also it obstructs the effort enhancing Flink cilent api,
> >> > > > > I'd like to propose dropping it. Or we can start a survey on
> user
> >> > list
> >> > > > > to see if there is any user depending on this class.
> >> > > > >
> >> > > > > best,
> >> > > > > tison.
> >> > > > >
> >> > > > > [1]
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> >> > > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: [DISCUSS] Drop stale class Program
Posted by Zili Chen <wa...@gmail.com>.
Thanks for your attentions!
Thank Kostas for creating the JIRA and drafting the FLIP.
I would volunteer to help review it :-)
It's good to see that we make progress on this thread.
Best,
tison.
Kostas Kloudas <kk...@gmail.com> 于2019年8月14日周三 下午6:39写道：
> I already opened a JIRA for the removal and I will also create a (short)
> FLIP, as it is a PublicEvolving interface and its removal should go through
> a FLIP.
>
> The JIRA can be found here
> https://issues.apache.org/jira/browse/FLINK-13713
>
> Cheers,
> Kostas
>
> On Wed, Aug 14, 2019 at 12:13 PM Stephan Ewen <se...@apache.org> wrote:
>
> > +1 to drop it.
> >
> > It's one of the oldest pieces of legacy.
> >
> > On Wed, Aug 14, 2019 at 12:07 PM Aljoscha Krettek <al...@apache.org>
> > wrote:
> >
> > > Hi,
> > >
> > > I would be in favour of removing Program (and the code paths that
> support
> > > it) for Flink 1.10. Most users of Flink don’t actually know it exists
> and
> > > it is only making our code more complicated. Going forward with the new
> > > Client API discussions will be a lot easier without it as well.
> > >
> > > Best,
> > > Aljoscha
> > >
> > > > On 14. Aug 2019, at 11:08, Kostas Kloudas <kk...@gmail.com>
> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > It is nice to have this discussion.
> > > >
> > > > I am totally up for removing the unused Program interface, as this
> will
> > > > simplify a lot of other code paths in the ClusterClient and
> elsewhere.
> > > >
> > > > Also about the easier integration of Flink with other frameworks,
> there
> > > > is another discussion in the mailing list with exactly this topic:
> > > > [DISCUSS] Flink client api enhancement for downstream project
> > > >
> > > > Cheers,
> > > > Kostas
> > > >
> > > >
> > > > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <wa...@gmail.com>
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> With a one-week survey in user list[1], nobody expect Flavio and
> Jeff
> > > >> participant the thread.
> > > >>
> > > >> Flavio shared his experience with a revised Program like interface.
> > > >> This could be regraded as downstream integration and in client api
> > > >> enhancements document we propose rich interface for this
> integration.
> > > >> Anyway, the Flink scope Program is less functional than it should
> be.
> > > >>
> > > >> With no objection I'd like to push on this thread. We need a
> committer
> > > >> participant this thread to shepherd the removal/deprecation of
> > Program,
> > > a
> > > >> @PublicEvolving interface. Anybody has spare time? Or anything I can
> > do
> > > >> to make progress?
> > > >>
> > > >> Best,
> > > >> tison.
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > > >>
> > > >>
> > > >> Zili Chen <wa...@gmail.com> 于2019年7月22日周一 下午8:38写道：
> > > >>
> > > >>> Hi,
> > > >>>
> > > >>> I created a thread for survey in user list[1]. Please take
> > participate
> > > in
> > > >>> if interested.
> > > >>>
> > > >>> Best,
> > > >>> tison.
> > > >>>
> > > >>> [1]
> > > >>>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > > >>>
> > > >>>
> > > >>> Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午5:18写道：
> > > >>>
> > > >>>> +1 to remove directly the Program class (I think nobody use it and
> > > it's
> > > >>>> not
> > > >>>> supported at all by REST services and UI).
> > > >>>> Moreover it requires a lot of transitive dependencies while it
> > should
> > > >> be a
> > > >>>> very simple thing..
> > > >>>> +1 to add this discussion to "Flink client api enhancement"
> > > >>>>
> > > >>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <mm...@gmail.com>
> > wrote:
> > > >>>>
> > > >>>>> To Flavio, good point for the integration suggestion.
> > > >>>>>
> > > >>>>> I think it should be considered in the "Flink client api
> > enhancement"
> > > >>>>> discussion. But the outdated API should be deprecated somehow.
> > > >>>>>
> > > >>>>> Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五
> 下午4:21写道：
> > > >>>>>
> > > >>>>>> In my experience a basic "official" (but optional) program
> > > >> description
> > > >>>>>> would be very useful indeed (in order to ease the integration
> with
> > > >>>> other
> > > >>>>>> frameworks).
> > > >>>>>>
> > > >>>>>> Of course it should be extended and integrated with the REST
> > > >> services
> > > >>>> and
> > > >>>>>> the Web UI (when defined) in order to be useful..
> > > >>>>>> It ease to show to the user what a job does and which parameters
> > it
> > > >>>>>> requires (optional or mandatory) and with a proper help
> > description.
> > > >>>>>> Indeed, when we write a Flink job we implement the following
> > > >>>> interface:
> > > >>>>>>
> > > >>>>>> public interface FlinkJob {
> > > >>>>>> String getDescription();
> > > >>>>>> List<FlinkJobParameter> getParameters();
> > > >>>>>> boolean isStreamingOrBatch();
> > > >>>>>> }
> > > >>>>>>
> > > >>>>>> public class ClusterJobParameter {
> > > >>>>>>
> > > >>>>>> private String paramName;
> > > >>>>>> private String paramType = "string";
> > > >>>>>> private String paramDesc;
> > > >>>>>> private String paramDefaultValue;
> > > >>>>>> private Set<String> choices;
> > > >>>>>> private boolean mandatory;
> > > >>>>>> }
> > > >>>>>>
> > > >>>>>> This really helps to launch a Flink job by a frontend (if the
> rest
> > > >>>>> services
> > > >>>>>> returns back those infos).
> > > >>>>>>
> > > >>>>>> Best,
> > > >>>>>> Flavio
> > > >>>>>>
> > > >>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <mm...@gmail.com>
> > > >> wrote:
> > > >>>>>>
> > > >>>>>>> Hi Zili,
> > > >>>>>>>
> > > >>>>>>> Thank you for bring us this discussion.
> > > >>>>>>>
> > > >>>>>>> My gut feeling is +1 for dropping it.
> > > >>>>>>> Usually it costs some time to deprecate a public (actually it's
> > > >>>>>>> `PublicEvolving`) API. Ideally it should be marked as
> > `Deprecated`
> > > >>>>> first.
> > > >>>>>>> Then it might be abandoned it in some later version.
> > > >>>>>>>
> > > >>>>>>> I'm not sure how big the burden is to make it compatible with
> the
> > > >>>>>> enhanced
> > > >>>>>>> client API. If it's a critical blocker, I support dropping it
> > > >>>> radically
> > > >>>>>> in
> > > >>>>>>> 1.10. Of course a survey is necessary. And the result of survey
> > is
> > > >>>>>>> acceptable.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Zili Chen <wa...@gmail.com> 于2019年7月19日周五 下午1:44写道：
> > > >>>>>>>
> > > >>>>>>>> Hi devs,
> > > >>>>>>>>
> > > >>>>>>>> Participating the thread "Flink client api enhancement"[1], I
> > > >> just
> > > >>>>>> notice
> > > >>>>>>>> that inside submission codepath of Flink we always has a
> branch
> > > >>>>> dealing
> > > >>>>>>>> with the case that main class of user program is a subclass of
> > > >>>>>>>> o.a.f.api.common.Program, which is defined as
> > > >>>>>>>>
> > > >>>>>>>> @PublicEvolving
> > > >>>>>>>> public interface Program {
> > > >>>>>>>> Plan getPhan(String... args);
> > > >>>>>>>> }
> > > >>>>>>>>
> > > >>>>>>>> This class, as user-facing interface, asks user to implement
> > > >>>> #getPlan
> > > >>>>>>>> which return a almost Flink internal class. FLINK-10862[2]
> > > >> pointed
> > > >>>>> out
> > > >>>>>>>> this confusion.
> > > >>>>>>>>
> > > >>>>>>>> Since our codebase contains quite a bit code handling this
> stale
> > > >>>>> class,
> > > >>>>>>>> and also it obstructs the effort enhancing Flink cilent api,
> > > >>>>>>>> I'd like to propose dropping it. Or we can start a survey on
> > > >> user
> > > >>>>> list
> > > >>>>>>>> to see if there is any user depending on this class.
> > > >>>>>>>>
> > > >>>>>>>> best,
> > > >>>>>>>> tison.
> > > >>>>>>>>
> > > >>>>>>>> [1]
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > >>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>
Re: [DISCUSS] Drop stale class Program
Posted by Kostas Kloudas <kk...@gmail.com>.
I already opened a JIRA for the removal and I will also create a (short)
FLIP, as it is a PublicEvolving interface and its removal should go through
a FLIP.
The JIRA can be found here https://issues.apache.org/jira/browse/FLINK-13713
Cheers,
Kostas
On Wed, Aug 14, 2019 at 12:13 PM Stephan Ewen <se...@apache.org> wrote:
> +1 to drop it.
>
> It's one of the oldest pieces of legacy.
>
> On Wed, Aug 14, 2019 at 12:07 PM Aljoscha Krettek <al...@apache.org>
> wrote:
>
> > Hi,
> >
> > I would be in favour of removing Program (and the code paths that support
> > it) for Flink 1.10. Most users of Flink don’t actually know it exists and
> > it is only making our code more complicated. Going forward with the new
> > Client API discussions will be a lot easier without it as well.
> >
> > Best,
> > Aljoscha
> >
> > > On 14. Aug 2019, at 11:08, Kostas Kloudas <kk...@gmail.com> wrote:
> > >
> > > Hi all,
> > >
> > > It is nice to have this discussion.
> > >
> > > I am totally up for removing the unused Program interface, as this will
> > > simplify a lot of other code paths in the ClusterClient and elsewhere.
> > >
> > > Also about the easier integration of Flink with other frameworks, there
> > > is another discussion in the mailing list with exactly this topic:
> > > [DISCUSS] Flink client api enhancement for downstream project
> > >
> > > Cheers,
> > > Kostas
> > >
> > >
> > > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <wa...@gmail.com>
> wrote:
> > >
> > >> Hi,
> > >>
> > >> With a one-week survey in user list[1], nobody expect Flavio and Jeff
> > >> participant the thread.
> > >>
> > >> Flavio shared his experience with a revised Program like interface.
> > >> This could be regraded as downstream integration and in client api
> > >> enhancements document we propose rich interface for this integration.
> > >> Anyway, the Flink scope Program is less functional than it should be.
> > >>
> > >> With no objection I'd like to push on this thread. We need a committer
> > >> participant this thread to shepherd the removal/deprecation of
> Program,
> > a
> > >> @PublicEvolving interface. Anybody has spare time? Or anything I can
> do
> > >> to make progress?
> > >>
> > >> Best,
> > >> tison.
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > >>
> > >>
> > >> Zili Chen <wa...@gmail.com> 于2019年7月22日周一 下午8:38写道：
> > >>
> > >>> Hi,
> > >>>
> > >>> I created a thread for survey in user list[1]. Please take
> participate
> > in
> > >>> if interested.
> > >>>
> > >>> Best,
> > >>> tison.
> > >>>
> > >>> [1]
> > >>>
> > >>
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > >>>
> > >>>
> > >>> Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午5:18写道：
> > >>>
> > >>>> +1 to remove directly the Program class (I think nobody use it and
> > it's
> > >>>> not
> > >>>> supported at all by REST services and UI).
> > >>>> Moreover it requires a lot of transitive dependencies while it
> should
> > >> be a
> > >>>> very simple thing..
> > >>>> +1 to add this discussion to "Flink client api enhancement"
> > >>>>
> > >>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <mm...@gmail.com>
> wrote:
> > >>>>
> > >>>>> To Flavio, good point for the integration suggestion.
> > >>>>>
> > >>>>> I think it should be considered in the "Flink client api
> enhancement"
> > >>>>> discussion. But the outdated API should be deprecated somehow.
> > >>>>>
> > >>>>> Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午4:21写道：
> > >>>>>
> > >>>>>> In my experience a basic "official" (but optional) program
> > >> description
> > >>>>>> would be very useful indeed (in order to ease the integration with
> > >>>> other
> > >>>>>> frameworks).
> > >>>>>>
> > >>>>>> Of course it should be extended and integrated with the REST
> > >> services
> > >>>> and
> > >>>>>> the Web UI (when defined) in order to be useful..
> > >>>>>> It ease to show to the user what a job does and which parameters
> it
> > >>>>>> requires (optional or mandatory) and with a proper help
> description.
> > >>>>>> Indeed, when we write a Flink job we implement the following
> > >>>> interface:
> > >>>>>>
> > >>>>>> public interface FlinkJob {
> > >>>>>> String getDescription();
> > >>>>>> List<FlinkJobParameter> getParameters();
> > >>>>>> boolean isStreamingOrBatch();
> > >>>>>> }
> > >>>>>>
> > >>>>>> public class ClusterJobParameter {
> > >>>>>>
> > >>>>>> private String paramName;
> > >>>>>> private String paramType = "string";
> > >>>>>> private String paramDesc;
> > >>>>>> private String paramDefaultValue;
> > >>>>>> private Set<String> choices;
> > >>>>>> private boolean mandatory;
> > >>>>>> }
> > >>>>>>
> > >>>>>> This really helps to launch a Flink job by a frontend (if the rest
> > >>>>> services
> > >>>>>> returns back those infos).
> > >>>>>>
> > >>>>>> Best,
> > >>>>>> Flavio
> > >>>>>>
> > >>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <mm...@gmail.com>
> > >> wrote:
> > >>>>>>
> > >>>>>>> Hi Zili,
> > >>>>>>>
> > >>>>>>> Thank you for bring us this discussion.
> > >>>>>>>
> > >>>>>>> My gut feeling is +1 for dropping it.
> > >>>>>>> Usually it costs some time to deprecate a public (actually it's
> > >>>>>>> `PublicEvolving`) API. Ideally it should be marked as
> `Deprecated`
> > >>>>> first.
> > >>>>>>> Then it might be abandoned it in some later version.
> > >>>>>>>
> > >>>>>>> I'm not sure how big the burden is to make it compatible with the
> > >>>>>> enhanced
> > >>>>>>> client API. If it's a critical blocker, I support dropping it
> > >>>> radically
> > >>>>>> in
> > >>>>>>> 1.10. Of course a survey is necessary. And the result of survey
> is
> > >>>>>>> acceptable.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Zili Chen <wa...@gmail.com> 于2019年7月19日周五 下午1:44写道：
> > >>>>>>>
> > >>>>>>>> Hi devs,
> > >>>>>>>>
> > >>>>>>>> Participating the thread "Flink client api enhancement"[1], I
> > >> just
> > >>>>>> notice
> > >>>>>>>> that inside submission codepath of Flink we always has a branch
> > >>>>> dealing
> > >>>>>>>> with the case that main class of user program is a subclass of
> > >>>>>>>> o.a.f.api.common.Program, which is defined as
> > >>>>>>>>
> > >>>>>>>> @PublicEvolving
> > >>>>>>>> public interface Program {
> > >>>>>>>> Plan getPhan(String... args);
> > >>>>>>>> }
> > >>>>>>>>
> > >>>>>>>> This class, as user-facing interface, asks user to implement
> > >>>> #getPlan
> > >>>>>>>> which return a almost Flink internal class. FLINK-10862[2]
> > >> pointed
> > >>>>> out
> > >>>>>>>> this confusion.
> > >>>>>>>>
> > >>>>>>>> Since our codebase contains quite a bit code handling this stale
> > >>>>> class,
> > >>>>>>>> and also it obstructs the effort enhancing Flink cilent api,
> > >>>>>>>> I'd like to propose dropping it. Or we can start a survey on
> > >> user
> > >>>>> list
> > >>>>>>>> to see if there is any user depending on this class.
> > >>>>>>>>
> > >>>>>>>> best,
> > >>>>>>>> tison.
> > >>>>>>>>
> > >>>>>>>> [1]
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > >>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>
Re: [DISCUSS] Drop stale class Program
Posted by Stephan Ewen <se...@apache.org>.
+1 to drop it.
It's one of the oldest pieces of legacy.
On Wed, Aug 14, 2019 at 12:07 PM Aljoscha Krettek <al...@apache.org>
wrote:
> Hi,
>
> I would be in favour of removing Program (and the code paths that support
> it) for Flink 1.10. Most users of Flink don’t actually know it exists and
> it is only making our code more complicated. Going forward with the new
> Client API discussions will be a lot easier without it as well.
>
> Best,
> Aljoscha
>
> > On 14. Aug 2019, at 11:08, Kostas Kloudas <kk...@gmail.com> wrote:
> >
> > Hi all,
> >
> > It is nice to have this discussion.
> >
> > I am totally up for removing the unused Program interface, as this will
> > simplify a lot of other code paths in the ClusterClient and elsewhere.
> >
> > Also about the easier integration of Flink with other frameworks, there
> > is another discussion in the mailing list with exactly this topic:
> > [DISCUSS] Flink client api enhancement for downstream project
> >
> > Cheers,
> > Kostas
> >
> >
> > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <wa...@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> With a one-week survey in user list[1], nobody expect Flavio and Jeff
> >> participant the thread.
> >>
> >> Flavio shared his experience with a revised Program like interface.
> >> This could be regraded as downstream integration and in client api
> >> enhancements document we propose rich interface for this integration.
> >> Anyway, the Flink scope Program is less functional than it should be.
> >>
> >> With no objection I'd like to push on this thread. We need a committer
> >> participant this thread to shepherd the removal/deprecation of Program,
> a
> >> @PublicEvolving interface. Anybody has spare time? Or anything I can do
> >> to make progress?
> >>
> >> Best,
> >> tison.
> >>
> >> [1]
> >>
> >>
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> >>
> >>
> >> Zili Chen <wa...@gmail.com> 于2019年7月22日周一 下午8:38写道：
> >>
> >>> Hi,
> >>>
> >>> I created a thread for survey in user list[1]. Please take participate
> in
> >>> if interested.
> >>>
> >>> Best,
> >>> tison.
> >>>
> >>> [1]
> >>>
> >>
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> >>>
> >>>
> >>> Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午5:18写道：
> >>>
> >>>> +1 to remove directly the Program class (I think nobody use it and
> it's
> >>>> not
> >>>> supported at all by REST services and UI).
> >>>> Moreover it requires a lot of transitive dependencies while it should
> >> be a
> >>>> very simple thing..
> >>>> +1 to add this discussion to "Flink client api enhancement"
> >>>>
> >>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <mm...@gmail.com> wrote:
> >>>>
> >>>>> To Flavio, good point for the integration suggestion.
> >>>>>
> >>>>> I think it should be considered in the "Flink client api enhancement"
> >>>>> discussion. But the outdated API should be deprecated somehow.
> >>>>>
> >>>>> Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午4:21写道：
> >>>>>
> >>>>>> In my experience a basic "official" (but optional) program
> >> description
> >>>>>> would be very useful indeed (in order to ease the integration with
> >>>> other
> >>>>>> frameworks).
> >>>>>>
> >>>>>> Of course it should be extended and integrated with the REST
> >> services
> >>>> and
> >>>>>> the Web UI (when defined) in order to be useful..
> >>>>>> It ease to show to the user what a job does and which parameters it
> >>>>>> requires (optional or mandatory) and with a proper help description.
> >>>>>> Indeed, when we write a Flink job we implement the following
> >>>> interface:
> >>>>>>
> >>>>>> public interface FlinkJob {
> >>>>>> String getDescription();
> >>>>>> List<FlinkJobParameter> getParameters();
> >>>>>> boolean isStreamingOrBatch();
> >>>>>> }
> >>>>>>
> >>>>>> public class ClusterJobParameter {
> >>>>>>
> >>>>>> private String paramName;
> >>>>>> private String paramType = "string";
> >>>>>> private String paramDesc;
> >>>>>> private String paramDefaultValue;
> >>>>>> private Set<String> choices;
> >>>>>> private boolean mandatory;
> >>>>>> }
> >>>>>>
> >>>>>> This really helps to launch a Flink job by a frontend (if the rest
> >>>>> services
> >>>>>> returns back those infos).
> >>>>>>
> >>>>>> Best,
> >>>>>> Flavio
> >>>>>>
> >>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <mm...@gmail.com>
> >> wrote:
> >>>>>>
> >>>>>>> Hi Zili,
> >>>>>>>
> >>>>>>> Thank you for bring us this discussion.
> >>>>>>>
> >>>>>>> My gut feeling is +1 for dropping it.
> >>>>>>> Usually it costs some time to deprecate a public (actually it's
> >>>>>>> `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> >>>>> first.
> >>>>>>> Then it might be abandoned it in some later version.
> >>>>>>>
> >>>>>>> I'm not sure how big the burden is to make it compatible with the
> >>>>>> enhanced
> >>>>>>> client API. If it's a critical blocker, I support dropping it
> >>>> radically
> >>>>>> in
> >>>>>>> 1.10. Of course a survey is necessary. And the result of survey is
> >>>>>>> acceptable.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Zili Chen <wa...@gmail.com> 于2019年7月19日周五 下午1:44写道：
> >>>>>>>
> >>>>>>>> Hi devs,
> >>>>>>>>
> >>>>>>>> Participating the thread "Flink client api enhancement"[1], I
> >> just
> >>>>>> notice
> >>>>>>>> that inside submission codepath of Flink we always has a branch
> >>>>> dealing
> >>>>>>>> with the case that main class of user program is a subclass of
> >>>>>>>> o.a.f.api.common.Program, which is defined as
> >>>>>>>>
> >>>>>>>> @PublicEvolving
> >>>>>>>> public interface Program {
> >>>>>>>> Plan getPhan(String... args);
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> This class, as user-facing interface, asks user to implement
> >>>> #getPlan
> >>>>>>>> which return a almost Flink internal class. FLINK-10862[2]
> >> pointed
> >>>>> out
> >>>>>>>> this confusion.
> >>>>>>>>
> >>>>>>>> Since our codebase contains quite a bit code handling this stale
> >>>>> class,
> >>>>>>>> and also it obstructs the effort enhancing Flink cilent api,
> >>>>>>>> I'd like to propose dropping it. Or we can start a survey on
> >> user
> >>>>> list
> >>>>>>>> to see if there is any user depending on this class.
> >>>>>>>>
> >>>>>>>> best,
> >>>>>>>> tison.
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> >>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
Re: [DISCUSS] Drop stale class Program
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
I would be in favour of removing Program (and the code paths that support it) for Flink 1.10. Most users of Flink don’t actually know it exists and it is only making our code more complicated. Going forward with the new Client API discussions will be a lot easier without it as well.
Best,
Aljoscha
> On 14. Aug 2019, at 11:08, Kostas Kloudas <kk...@gmail.com> wrote:
>
> Hi all,
>
> It is nice to have this discussion.
>
> I am totally up for removing the unused Program interface, as this will
> simplify a lot of other code paths in the ClusterClient and elsewhere.
>
> Also about the easier integration of Flink with other frameworks, there
> is another discussion in the mailing list with exactly this topic:
> [DISCUSS] Flink client api enhancement for downstream project
>
> Cheers,
> Kostas
>
>
> On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <wa...@gmail.com> wrote:
>
>> Hi,
>>
>> With a one-week survey in user list[1], nobody expect Flavio and Jeff
>> participant the thread.
>>
>> Flavio shared his experience with a revised Program like interface.
>> This could be regraded as downstream integration and in client api
>> enhancements document we propose rich interface for this integration.
>> Anyway, the Flink scope Program is less functional than it should be.
>>
>> With no objection I'd like to push on this thread. We need a committer
>> participant this thread to shepherd the removal/deprecation of Program, a
>> @PublicEvolving interface. Anybody has spare time? Or anything I can do
>> to make progress?
>>
>> Best,
>> tison.
>>
>> [1]
>>
>> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>>
>>
>> Zili Chen <wa...@gmail.com> 于2019年7月22日周一 下午8:38写道：
>>
>>> Hi,
>>>
>>> I created a thread for survey in user list[1]. Please take participate in
>>> if interested.
>>>
>>> Best,
>>> tison.
>>>
>>> [1]
>>>
>> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>>>
>>>
>>> Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午5:18写道：
>>>
>>>> +1 to remove directly the Program class (I think nobody use it and it's
>>>> not
>>>> supported at all by REST services and UI).
>>>> Moreover it requires a lot of transitive dependencies while it should
>> be a
>>>> very simple thing..
>>>> +1 to add this discussion to "Flink client api enhancement"
>>>>
>>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <mm...@gmail.com> wrote:
>>>>
>>>>> To Flavio, good point for the integration suggestion.
>>>>>
>>>>> I think it should be considered in the "Flink client api enhancement"
>>>>> discussion. But the outdated API should be deprecated somehow.
>>>>>
>>>>> Flavio Pompermaier <po...@okkam.it> 于2019年7月19日周五 下午4:21写道：
>>>>>
>>>>>> In my experience a basic "official" (but optional) program
>> description
>>>>>> would be very useful indeed (in order to ease the integration with
>>>> other
>>>>>> frameworks).
>>>>>>
>>>>>> Of course it should be extended and integrated with the REST
>> services
>>>> and
>>>>>> the Web UI (when defined) in order to be useful..
>>>>>> It ease to show to the user what a job does and which parameters it
>>>>>> requires (optional or mandatory) and with a proper help description.
>>>>>> Indeed, when we write a Flink job we implement the following
>>>> interface:
>>>>>>
>>>>>> public interface FlinkJob {
>>>>>> String getDescription();
>>>>>> List<FlinkJobParameter> getParameters();
>>>>>> boolean isStreamingOrBatch();
>>>>>> }
>>>>>>
>>>>>> public class ClusterJobParameter {
>>>>>>
>>>>>> private String paramName;
>>>>>> private String paramType = "string";
>>>>>> private String paramDesc;
>>>>>> private String paramDefaultValue;
>>>>>> private Set<String> choices;
>>>>>> private boolean mandatory;
>>>>>> }
>>>>>>
>>>>>> This really helps to launch a Flink job by a frontend (if the rest
>>>>> services
>>>>>> returns back those infos).
>>>>>>
>>>>>> Best,
>>>>>> Flavio
>>>>>>
>>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <mm...@gmail.com>
>> wrote:
>>>>>>
>>>>>>> Hi Zili,
>>>>>>>
>>>>>>> Thank you for bring us this discussion.
>>>>>>>
>>>>>>> My gut feeling is +1 for dropping it.
>>>>>>> Usually it costs some time to deprecate a public (actually it's
>>>>>>> `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
>>>>> first.
>>>>>>> Then it might be abandoned it in some later version.
>>>>>>>
>>>>>>> I'm not sure how big the burden is to make it compatible with the
>>>>>> enhanced
>>>>>>> client API. If it's a critical blocker, I support dropping it
>>>> radically
>>>>>> in
>>>>>>> 1.10. Of course a survey is necessary. And the result of survey is
>>>>>>> acceptable.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Zili Chen <wa...@gmail.com> 于2019年7月19日周五 下午1:44写道：
>>>>>>>
>>>>>>>> Hi devs,
>>>>>>>>
>>>>>>>> Participating the thread "Flink client api enhancement"[1], I
>> just
>>>>>> notice
>>>>>>>> that inside submission codepath of Flink we always has a branch
>>>>> dealing
>>>>>>>> with the case that main class of user program is a subclass of
>>>>>>>> o.a.f.api.common.Program, which is defined as
>>>>>>>>
>>>>>>>> @PublicEvolving
>>>>>>>> public interface Program {
>>>>>>>> Plan getPhan(String... args);
>>>>>>>> }
>>>>>>>>
>>>>>>>> This class, as user-facing interface, asks user to implement
>>>> #getPlan
>>>>>>>> which return a almost Flink internal class. FLINK-10862[2]
>> pointed
>>>>> out
>>>>>>>> this confusion.
>>>>>>>>
>>>>>>>> Since our codebase contains quite a bit code handling this stale
>>>>> class,
>>>>>>>> and also it obstructs the effort enhancing Flink cilent api,
>>>>>>>> I'd like to propose dropping it. Or we can start a survey on
>> user
>>>>> list
>>>>>>>> to see if there is any user depending on this class.
>>>>>>>>
>>>>>>>> best,
>>>>>>>> tison.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
>>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>