You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Gabor Somogyi <ga...@gmail.com> on 2022/06/16 10:43:54 UTC
[DISCUSS] How to delete deprecated configs/functions
Hi All,
I've just tried to delete long time deprecated configs and faced some
inconveniences.
My main intention is to understand what was the idea
when japicmp-maven-plugin introduced related configs/functions marked
with @Deprecated annotation.
When I've removed the mentioned configs from source code then
japicmp-maven-plugin made the build failed and complained that the params
are removed which makes sense.
I've added @Deprecated annotation to excludes which made the original error
disappear but new ones shown because with that exclude all new version
annotated configs/functions are missing.
This makes this solution dead end.
The only solution I've found is to add the specific configs to the exclude
list. It works like charm but adding all the upcoming removed deprecated
configs/functions to the exclude list will end-up in a constantly growing
exclude list.
Just to give a concrete example: https://github.com/apache/flink/pull/19986
Is there another way which I've missed or this is going to be hot issue
later on?
Thanks in advance!
BR,
G
Re: [DISCUSS] How to delete deprecated configs/functions
Posted by Chesnay Schepler <ch...@apache.org>.
a) Nothing in there was @Public afaict
b) even if a) weren't the case, we voted on it
c) there was an appreciable benefit in dropping Mesos, since it paved
the way for Scala-free Flink
On 16/06/2022 12:59, Gabor Somogyi wrote:
> OK, I see your point.
>
> Just a question here. Mesos is deprecated in 1.13 here:
> https://issues.apache.org/jira/browse/FLINK-22352
> and now I don't see any Mesos related classes in 1.16-SNAPSHOT.
>
> No matter from which angle I take a look at it this is breaking
> change. What makes the 2 cases different?
>
> G
>
>
> On Thu, Jun 16, 2022 at 12:49 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> The japicm-plugin did exactly what it was supposed to be doing.
>
> We can't remove these fields because they are part of a @Public class
> and hence have to remain until Flink 2.0.
>
> Adding an exclusion for @Deprecated is a bad idea because it makes it
> way to easy to break the API.
>
> On 16/06/2022 12:43, Gabor Somogyi wrote:
> > Hi All,
> >
> > I've just tried to delete long time deprecated configs and faced
> some
> > inconveniences.
> > My main intention is to understand what was the idea
> > when japicmp-maven-plugin introduced related configs/functions
> marked
> > with @Deprecated annotation.
> >
> > When I've removed the mentioned configs from source code then
> > japicmp-maven-plugin made the build failed and complained that
> the params
> > are removed which makes sense.
> >
> > I've added @Deprecated annotation to excludes which made the
> original error
> > disappear but new ones shown because with that exclude all new
> version
> > annotated configs/functions are missing.
> > This makes this solution dead end.
> >
> > The only solution I've found is to add the specific configs to
> the exclude
> > list. It works like charm but adding all the upcoming removed
> deprecated
> > configs/functions to the exclude list will end-up in a
> constantly growing
> > exclude list.
> >
> > Just to give a concrete example:
> https://github.com/apache/flink/pull/19986
> >
> > Is there another way which I've missed or this is going to be
> hot issue
> > later on?
> >
> > Thanks in advance!
> >
> > BR,
> > G
> >
>
Re: [DISCUSS] How to delete deprecated configs/functions
Posted by Gabor Somogyi <ga...@gmail.com>.
Now I see the difference. All in all the main reason of this discussion is
that I've found many super old Hadoop workarounds/unused configs/not
working solutions.
I think it would be beneficial for the community to remove/fix them
somehow. There are 4-7 years old issues inside...
This is a good example apart from the unused and long time deprecated
configs but I can mention further things:
https://github.com/apache/flink/blob/8d05393f5bcc0a917b2dab3fe81a58acaccabf13/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java#L157-L159
If you guys are open to such intention then we can discuss the details.
G
On Thu, Jun 16, 2022 at 1:14 PM Gyula Fóra <gy...@gmail.com> wrote:
> In any case these configs are related to supporting some very very old
> Hadoop versions. Which are likely broken anyways.
> I think we can also vote on removing support for some of these keys
> together with discussing (and voting on) the minimum supported hadoop
> version.
>
> In other words I think we should follow the previous practices when we
> removed support for other connector versions, modules etc.
>
> Gyula
>
> On Thu, Jun 16, 2022 at 1:08 PM Gyula Fóra <gy...@gmail.com> wrote:
>
>> Chesnay,
>>
>> I think as Gabor said, we have removed modules and functionality in the
>> past which basically made some old configuration keys useless (effectively
>> removed).
>> In this particular case we don't need to remove the static constants that
>> store this config key, we can simply ignore them. This would leave some
>> legacy garbage there but would not break the japicmp plugin.
>>
>> In any case I think dropping support for this legacy config key makes
>> sense (even if it remains in the code if we insist on it).
>>
>> Gyula
>>
>>
>> On Thu, Jun 16, 2022 at 1:05 PM Chesnay Schepler <ch...@apache.org>
>> wrote:
>>
>>> d) The japicmp plugin is about programmatic API compatibility, which
>>> doesn't cover Mesos support.
>>>
>>> On 16/06/2022 12:59, Gabor Somogyi wrote:
>>> > OK, I see your point.
>>> >
>>> > Just a question here. Mesos is deprecated in 1.13 here:
>>> > https://issues.apache.org/jira/browse/FLINK-22352
>>> > and now I don't see any Mesos related classes in 1.16-SNAPSHOT.
>>> >
>>> > No matter from which angle I take a look at it this is breaking change.
>>> > What makes the 2 cases different?
>>> >
>>> > G
>>> >
>>> >
>>> > On Thu, Jun 16, 2022 at 12:49 PM Chesnay Schepler <ch...@apache.org>
>>> > wrote:
>>> >
>>> >> The japicm-plugin did exactly what it was supposed to be doing.
>>> >>
>>> >> We can't remove these fields because they are part of a @Public class
>>> >> and hence have to remain until Flink 2.0.
>>> >>
>>> >> Adding an exclusion for @Deprecated is a bad idea because it makes it
>>> >> way to easy to break the API.
>>> >>
>>> >> On 16/06/2022 12:43, Gabor Somogyi wrote:
>>> >>> Hi All,
>>> >>>
>>> >>> I've just tried to delete long time deprecated configs and faced some
>>> >>> inconveniences.
>>> >>> My main intention is to understand what was the idea
>>> >>> when japicmp-maven-plugin introduced related configs/functions marked
>>> >>> with @Deprecated annotation.
>>> >>>
>>> >>> When I've removed the mentioned configs from source code then
>>> >>> japicmp-maven-plugin made the build failed and complained that the
>>> params
>>> >>> are removed which makes sense.
>>> >>>
>>> >>> I've added @Deprecated annotation to excludes which made the original
>>> >> error
>>> >>> disappear but new ones shown because with that exclude all new
>>> version
>>> >>> annotated configs/functions are missing.
>>> >>> This makes this solution dead end.
>>> >>>
>>> >>> The only solution I've found is to add the specific configs to the
>>> >> exclude
>>> >>> list. It works like charm but adding all the upcoming removed
>>> deprecated
>>> >>> configs/functions to the exclude list will end-up in a constantly
>>> growing
>>> >>> exclude list.
>>> >>>
>>> >>> Just to give a concrete example:
>>> >> https://github.com/apache/flink/pull/19986
>>> >>> Is there another way which I've missed or this is going to be hot
>>> issue
>>> >>> later on?
>>> >>>
>>> >>> Thanks in advance!
>>> >>>
>>> >>> BR,
>>> >>> G
>>> >>>
>>> >>
>>>
>>>
Re: [DISCUSS] How to delete deprecated configs/functions
Posted by Gyula Fóra <gy...@gmail.com>.
In any case these configs are related to supporting some very very old
Hadoop versions. Which are likely broken anyways.
I think we can also vote on removing support for some of these keys
together with discussing (and voting on) the minimum supported hadoop
version.
In other words I think we should follow the previous practices when we
removed support for other connector versions, modules etc.
Gyula
On Thu, Jun 16, 2022 at 1:08 PM Gyula Fóra <gy...@gmail.com> wrote:
> Chesnay,
>
> I think as Gabor said, we have removed modules and functionality in the
> past which basically made some old configuration keys useless (effectively
> removed).
> In this particular case we don't need to remove the static constants that
> store this config key, we can simply ignore them. This would leave some
> legacy garbage there but would not break the japicmp plugin.
>
> In any case I think dropping support for this legacy config key makes
> sense (even if it remains in the code if we insist on it).
>
> Gyula
>
>
> On Thu, Jun 16, 2022 at 1:05 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
>> d) The japicmp plugin is about programmatic API compatibility, which
>> doesn't cover Mesos support.
>>
>> On 16/06/2022 12:59, Gabor Somogyi wrote:
>> > OK, I see your point.
>> >
>> > Just a question here. Mesos is deprecated in 1.13 here:
>> > https://issues.apache.org/jira/browse/FLINK-22352
>> > and now I don't see any Mesos related classes in 1.16-SNAPSHOT.
>> >
>> > No matter from which angle I take a look at it this is breaking change.
>> > What makes the 2 cases different?
>> >
>> > G
>> >
>> >
>> > On Thu, Jun 16, 2022 at 12:49 PM Chesnay Schepler <ch...@apache.org>
>> > wrote:
>> >
>> >> The japicm-plugin did exactly what it was supposed to be doing.
>> >>
>> >> We can't remove these fields because they are part of a @Public class
>> >> and hence have to remain until Flink 2.0.
>> >>
>> >> Adding an exclusion for @Deprecated is a bad idea because it makes it
>> >> way to easy to break the API.
>> >>
>> >> On 16/06/2022 12:43, Gabor Somogyi wrote:
>> >>> Hi All,
>> >>>
>> >>> I've just tried to delete long time deprecated configs and faced some
>> >>> inconveniences.
>> >>> My main intention is to understand what was the idea
>> >>> when japicmp-maven-plugin introduced related configs/functions marked
>> >>> with @Deprecated annotation.
>> >>>
>> >>> When I've removed the mentioned configs from source code then
>> >>> japicmp-maven-plugin made the build failed and complained that the
>> params
>> >>> are removed which makes sense.
>> >>>
>> >>> I've added @Deprecated annotation to excludes which made the original
>> >> error
>> >>> disappear but new ones shown because with that exclude all new version
>> >>> annotated configs/functions are missing.
>> >>> This makes this solution dead end.
>> >>>
>> >>> The only solution I've found is to add the specific configs to the
>> >> exclude
>> >>> list. It works like charm but adding all the upcoming removed
>> deprecated
>> >>> configs/functions to the exclude list will end-up in a constantly
>> growing
>> >>> exclude list.
>> >>>
>> >>> Just to give a concrete example:
>> >> https://github.com/apache/flink/pull/19986
>> >>> Is there another way which I've missed or this is going to be hot
>> issue
>> >>> later on?
>> >>>
>> >>> Thanks in advance!
>> >>>
>> >>> BR,
>> >>> G
>> >>>
>> >>
>>
>>
Re: [DISCUSS] How to delete deprecated configs/functions
Posted by Gyula Fóra <gy...@gmail.com>.
Chesnay,
I think as Gabor said, we have removed modules and functionality in the
past which basically made some old configuration keys useless (effectively
removed).
In this particular case we don't need to remove the static constants that
store this config key, we can simply ignore them. This would leave some
legacy garbage there but would not break the japicmp plugin.
In any case I think dropping support for this legacy config key makes sense
(even if it remains in the code if we insist on it).
Gyula
On Thu, Jun 16, 2022 at 1:05 PM Chesnay Schepler <ch...@apache.org> wrote:
> d) The japicmp plugin is about programmatic API compatibility, which
> doesn't cover Mesos support.
>
> On 16/06/2022 12:59, Gabor Somogyi wrote:
> > OK, I see your point.
> >
> > Just a question here. Mesos is deprecated in 1.13 here:
> > https://issues.apache.org/jira/browse/FLINK-22352
> > and now I don't see any Mesos related classes in 1.16-SNAPSHOT.
> >
> > No matter from which angle I take a look at it this is breaking change.
> > What makes the 2 cases different?
> >
> > G
> >
> >
> > On Thu, Jun 16, 2022 at 12:49 PM Chesnay Schepler <ch...@apache.org>
> > wrote:
> >
> >> The japicm-plugin did exactly what it was supposed to be doing.
> >>
> >> We can't remove these fields because they are part of a @Public class
> >> and hence have to remain until Flink 2.0.
> >>
> >> Adding an exclusion for @Deprecated is a bad idea because it makes it
> >> way to easy to break the API.
> >>
> >> On 16/06/2022 12:43, Gabor Somogyi wrote:
> >>> Hi All,
> >>>
> >>> I've just tried to delete long time deprecated configs and faced some
> >>> inconveniences.
> >>> My main intention is to understand what was the idea
> >>> when japicmp-maven-plugin introduced related configs/functions marked
> >>> with @Deprecated annotation.
> >>>
> >>> When I've removed the mentioned configs from source code then
> >>> japicmp-maven-plugin made the build failed and complained that the
> params
> >>> are removed which makes sense.
> >>>
> >>> I've added @Deprecated annotation to excludes which made the original
> >> error
> >>> disappear but new ones shown because with that exclude all new version
> >>> annotated configs/functions are missing.
> >>> This makes this solution dead end.
> >>>
> >>> The only solution I've found is to add the specific configs to the
> >> exclude
> >>> list. It works like charm but adding all the upcoming removed
> deprecated
> >>> configs/functions to the exclude list will end-up in a constantly
> growing
> >>> exclude list.
> >>>
> >>> Just to give a concrete example:
> >> https://github.com/apache/flink/pull/19986
> >>> Is there another way which I've missed or this is going to be hot issue
> >>> later on?
> >>>
> >>> Thanks in advance!
> >>>
> >>> BR,
> >>> G
> >>>
> >>
>
>
Re: [DISCUSS] How to delete deprecated configs/functions
Posted by Chesnay Schepler <ch...@apache.org>.
d) The japicmp plugin is about programmatic API compatibility, which
doesn't cover Mesos support.
On 16/06/2022 12:59, Gabor Somogyi wrote:
> OK, I see your point.
>
> Just a question here. Mesos is deprecated in 1.13 here:
> https://issues.apache.org/jira/browse/FLINK-22352
> and now I don't see any Mesos related classes in 1.16-SNAPSHOT.
>
> No matter from which angle I take a look at it this is breaking change.
> What makes the 2 cases different?
>
> G
>
>
> On Thu, Jun 16, 2022 at 12:49 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
>> The japicm-plugin did exactly what it was supposed to be doing.
>>
>> We can't remove these fields because they are part of a @Public class
>> and hence have to remain until Flink 2.0.
>>
>> Adding an exclusion for @Deprecated is a bad idea because it makes it
>> way to easy to break the API.
>>
>> On 16/06/2022 12:43, Gabor Somogyi wrote:
>>> Hi All,
>>>
>>> I've just tried to delete long time deprecated configs and faced some
>>> inconveniences.
>>> My main intention is to understand what was the idea
>>> when japicmp-maven-plugin introduced related configs/functions marked
>>> with @Deprecated annotation.
>>>
>>> When I've removed the mentioned configs from source code then
>>> japicmp-maven-plugin made the build failed and complained that the params
>>> are removed which makes sense.
>>>
>>> I've added @Deprecated annotation to excludes which made the original
>> error
>>> disappear but new ones shown because with that exclude all new version
>>> annotated configs/functions are missing.
>>> This makes this solution dead end.
>>>
>>> The only solution I've found is to add the specific configs to the
>> exclude
>>> list. It works like charm but adding all the upcoming removed deprecated
>>> configs/functions to the exclude list will end-up in a constantly growing
>>> exclude list.
>>>
>>> Just to give a concrete example:
>> https://github.com/apache/flink/pull/19986
>>> Is there another way which I've missed or this is going to be hot issue
>>> later on?
>>>
>>> Thanks in advance!
>>>
>>> BR,
>>> G
>>>
>>
Re: [DISCUSS] How to delete deprecated configs/functions
Posted by Gabor Somogyi <ga...@gmail.com>.
OK, I see your point.
Just a question here. Mesos is deprecated in 1.13 here:
https://issues.apache.org/jira/browse/FLINK-22352
and now I don't see any Mesos related classes in 1.16-SNAPSHOT.
No matter from which angle I take a look at it this is breaking change.
What makes the 2 cases different?
G
On Thu, Jun 16, 2022 at 12:49 PM Chesnay Schepler <ch...@apache.org>
wrote:
> The japicm-plugin did exactly what it was supposed to be doing.
>
> We can't remove these fields because they are part of a @Public class
> and hence have to remain until Flink 2.0.
>
> Adding an exclusion for @Deprecated is a bad idea because it makes it
> way to easy to break the API.
>
> On 16/06/2022 12:43, Gabor Somogyi wrote:
> > Hi All,
> >
> > I've just tried to delete long time deprecated configs and faced some
> > inconveniences.
> > My main intention is to understand what was the idea
> > when japicmp-maven-plugin introduced related configs/functions marked
> > with @Deprecated annotation.
> >
> > When I've removed the mentioned configs from source code then
> > japicmp-maven-plugin made the build failed and complained that the params
> > are removed which makes sense.
> >
> > I've added @Deprecated annotation to excludes which made the original
> error
> > disappear but new ones shown because with that exclude all new version
> > annotated configs/functions are missing.
> > This makes this solution dead end.
> >
> > The only solution I've found is to add the specific configs to the
> exclude
> > list. It works like charm but adding all the upcoming removed deprecated
> > configs/functions to the exclude list will end-up in a constantly growing
> > exclude list.
> >
> > Just to give a concrete example:
> https://github.com/apache/flink/pull/19986
> >
> > Is there another way which I've missed or this is going to be hot issue
> > later on?
> >
> > Thanks in advance!
> >
> > BR,
> > G
> >
>
>
Re: [DISCUSS] How to delete deprecated configs/functions
Posted by Chesnay Schepler <ch...@apache.org>.
The japicm-plugin did exactly what it was supposed to be doing.
We can't remove these fields because they are part of a @Public class
and hence have to remain until Flink 2.0.
Adding an exclusion for @Deprecated is a bad idea because it makes it
way to easy to break the API.
On 16/06/2022 12:43, Gabor Somogyi wrote:
> Hi All,
>
> I've just tried to delete long time deprecated configs and faced some
> inconveniences.
> My main intention is to understand what was the idea
> when japicmp-maven-plugin introduced related configs/functions marked
> with @Deprecated annotation.
>
> When I've removed the mentioned configs from source code then
> japicmp-maven-plugin made the build failed and complained that the params
> are removed which makes sense.
>
> I've added @Deprecated annotation to excludes which made the original error
> disappear but new ones shown because with that exclude all new version
> annotated configs/functions are missing.
> This makes this solution dead end.
>
> The only solution I've found is to add the specific configs to the exclude
> list. It works like charm but adding all the upcoming removed deprecated
> configs/functions to the exclude list will end-up in a constantly growing
> exclude list.
>
> Just to give a concrete example: https://github.com/apache/flink/pull/19986
>
> Is there another way which I've missed or this is going to be hot issue
> later on?
>
> Thanks in advance!
>
> BR,
> G
>