You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Chesnay Schepler <ch...@apache.org> on 2021/12/06 13:00:35 UTC
Re: [DISCUSS] Drop Zookeeper 3.4
ping @users; any input on how this would affect you is highly appreciated.
On 25/11/2021 22:39, Chesnay Schepler wrote:
> I included the user ML in the thread.
>
> @users Are you still using Zookeeper 3.4? If so, were you planning to
> upgrade Zookeeper in the near future?
>
> I'm not sure about ZK compatibility, but we'd also upgrade Curator to
> 5.x, which doesn't support ookeeperK 3.4 anymore.
>
> On 25/11/2021 21:56, Till Rohrmann wrote:
>> Should we ask on the user mailing list whether anybody is still using
>> ZooKeeper 3.4 and thus needs support for this version or can a ZooKeeper
>> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that
>> not a
>> lot of users depend on it but just to make sure that we aren't
>> annoying a
>> lot of our users with this change. Apart from that +1 for removing it if
>> not a lot of user depend on it.
>>
>> Cheers,
>> Till
>>
>> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl <ma...@ververica.com>
>> wrote:
>>
>>> Thanks for starting this discussion, Chesnay. +1 from my side. It's
>>> time to
>>> move forward with the ZK support considering the EOL of 3.4 you already
>>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
>>> consequence is another plus point. Just for reference on the
>>> inconsistent
>>> state issue you mentioned: FLINK-24543 [1].
>>>
>>> Matthias
>>>
>>> [1] https://issues.apache.org/jira/browse/FLINK-24543
>>>
>>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler <ch...@apache.org>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
>>>> default to 3.5 with an opt-in for 3.6.
>>>>
>>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
>>>> upgrading Curator to 5.x, which would allow us to properly fix an
>>>> issue
>>>> with inconsistent state. It is also required to eventually support ZK
>>> 3.6.
>
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Yang Wang <da...@gmail.com>.
After FLINK-10052[1], which was merged in 1.14.0, rolling upgrading
ZooKeeper will not affect the running Flink application.
1. https://issues.apache.org/jira/browse/FLINK-10052
Best,
Yang
Chesnay Schepler <ch...@apache.org> 于2021年12月7日周二 下午4:37写道：
> Since this is only relevant for 1.15, if you intend to migrate to 1.15
> close to the release, then somewhere around February.
>
> The only resource I could find for migrating Zookeeper is this FAQ:
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ
>
> On 07/12/2021 04:02, Dongwon Kim wrote:
>
> When should I prepare for upgrading ZK to 3.5 or newer?
> We're operating a Hadoop cluster w/ ZK 3.4.6 for running only Flink jobs.
> Just hope that the rolling update is not that painful - any advice on this?
>
> Best,
>
> Dongwon
>
> On Tue, Dec 7, 2021 at 3:22 AM Chesnay Schepler <ch...@apache.org>
> wrote:
>
>> Current users of ZK 3.4 and below would need to upgrade their Zookeeper
>> installation that is used by Flink to 3.5+.
>>
>> Whether K8s users are affected depends on whether they use ZK or not. If
>> they do, see above, otherwise they are not affected at all.
>>
>> On 06/12/2021 18:49, Arvid Heise wrote:
>>
>> Could someone please help me understand the implications of the upgrade?
>>
>> As far as I understood this upgrade would only affect users that have a
>> zookeeper shared across multiple services, some of which require ZK 3.4-? A
>> workaround for those users would be to run two ZKs with different versions,
>> eventually deprecating old ZK, correct?
>>
>> If that is the only limitation, I'm +1 for the proposal since ZK 3.4 is
>> already EOL.
>>
>> How are K8s users affected?
>>
>> Best,
>>
>> Arvid
>>
>> On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org>
>> wrote:
>>
>>> ping @users; any input on how this would affect you is highly
>>> appreciated.
>>>
>>> On 25/11/2021 22:39, Chesnay Schepler wrote:
>>> > I included the user ML in the thread.
>>> >
>>> > @users Are you still using Zookeeper 3.4? If so, were you planning to
>>> > upgrade Zookeeper in the near future?
>>> >
>>> > I'm not sure about ZK compatibility, but we'd also upgrade Curator to
>>> > 5.x, which doesn't support ookeeperK 3.4 anymore.
>>> >
>>> > On 25/11/2021 21:56, Till Rohrmann wrote:
>>> >> Should we ask on the user mailing list whether anybody is still using
>>> >> ZooKeeper 3.4 and thus needs support for this version or can a
>>> ZooKeeper
>>> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that
>>> >> not a
>>> >> lot of users depend on it but just to make sure that we aren't
>>> >> annoying a
>>> >> lot of our users with this change. Apart from that +1 for removing it
>>> if
>>> >> not a lot of user depend on it.
>>> >>
>>> >> Cheers,
>>> >> Till
>>> >>
>>> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl <
>>> matthias@ververica.com>
>>> >> wrote:
>>> >>
>>> >>> Thanks for starting this discussion, Chesnay. +1 from my side. It's
>>> >>> time to
>>> >>> move forward with the ZK support considering the EOL of 3.4 you
>>> already
>>> >>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
>>> >>> consequence is another plus point. Just for reference on the
>>> >>> inconsistent
>>> >>> state issue you mentioned: FLINK-24543 [1].
>>> >>>
>>> >>> Matthias
>>> >>>
>>> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
>>> >>>
>>> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler <
>>> chesnay@apache.org>
>>> >>> wrote:
>>> >>>
>>> >>>> Hello,
>>> >>>>
>>> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
>>> >>>> default to 3.5 with an opt-in for 3.6.
>>> >>>>
>>> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
>>> >>>> upgrading Curator to 5.x, which would allow us to properly fix an
>>> >>>> issue
>>> >>>> with inconsistent state. It is also required to eventually support
>>> ZK
>>> >>> 3.6.
>>> >
>>> >
>>>
>>>
>>
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Yang Wang <da...@gmail.com>.
After FLINK-10052[1], which was merged in 1.14.0, rolling upgrading
ZooKeeper will not affect the running Flink application.
1. https://issues.apache.org/jira/browse/FLINK-10052
Best,
Yang
Chesnay Schepler <ch...@apache.org> 于2021年12月7日周二 下午4:37写道：
> Since this is only relevant for 1.15, if you intend to migrate to 1.15
> close to the release, then somewhere around February.
>
> The only resource I could find for migrating Zookeeper is this FAQ:
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ
>
> On 07/12/2021 04:02, Dongwon Kim wrote:
>
> When should I prepare for upgrading ZK to 3.5 or newer?
> We're operating a Hadoop cluster w/ ZK 3.4.6 for running only Flink jobs.
> Just hope that the rolling update is not that painful - any advice on this?
>
> Best,
>
> Dongwon
>
> On Tue, Dec 7, 2021 at 3:22 AM Chesnay Schepler <ch...@apache.org>
> wrote:
>
>> Current users of ZK 3.4 and below would need to upgrade their Zookeeper
>> installation that is used by Flink to 3.5+.
>>
>> Whether K8s users are affected depends on whether they use ZK or not. If
>> they do, see above, otherwise they are not affected at all.
>>
>> On 06/12/2021 18:49, Arvid Heise wrote:
>>
>> Could someone please help me understand the implications of the upgrade?
>>
>> As far as I understood this upgrade would only affect users that have a
>> zookeeper shared across multiple services, some of which require ZK 3.4-? A
>> workaround for those users would be to run two ZKs with different versions,
>> eventually deprecating old ZK, correct?
>>
>> If that is the only limitation, I'm +1 for the proposal since ZK 3.4 is
>> already EOL.
>>
>> How are K8s users affected?
>>
>> Best,
>>
>> Arvid
>>
>> On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org>
>> wrote:
>>
>>> ping @users; any input on how this would affect you is highly
>>> appreciated.
>>>
>>> On 25/11/2021 22:39, Chesnay Schepler wrote:
>>> > I included the user ML in the thread.
>>> >
>>> > @users Are you still using Zookeeper 3.4? If so, were you planning to
>>> > upgrade Zookeeper in the near future?
>>> >
>>> > I'm not sure about ZK compatibility, but we'd also upgrade Curator to
>>> > 5.x, which doesn't support ookeeperK 3.4 anymore.
>>> >
>>> > On 25/11/2021 21:56, Till Rohrmann wrote:
>>> >> Should we ask on the user mailing list whether anybody is still using
>>> >> ZooKeeper 3.4 and thus needs support for this version or can a
>>> ZooKeeper
>>> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that
>>> >> not a
>>> >> lot of users depend on it but just to make sure that we aren't
>>> >> annoying a
>>> >> lot of our users with this change. Apart from that +1 for removing it
>>> if
>>> >> not a lot of user depend on it.
>>> >>
>>> >> Cheers,
>>> >> Till
>>> >>
>>> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl <
>>> matthias@ververica.com>
>>> >> wrote:
>>> >>
>>> >>> Thanks for starting this discussion, Chesnay. +1 from my side. It's
>>> >>> time to
>>> >>> move forward with the ZK support considering the EOL of 3.4 you
>>> already
>>> >>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
>>> >>> consequence is another plus point. Just for reference on the
>>> >>> inconsistent
>>> >>> state issue you mentioned: FLINK-24543 [1].
>>> >>>
>>> >>> Matthias
>>> >>>
>>> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
>>> >>>
>>> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler <
>>> chesnay@apache.org>
>>> >>> wrote:
>>> >>>
>>> >>>> Hello,
>>> >>>>
>>> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
>>> >>>> default to 3.5 with an opt-in for 3.6.
>>> >>>>
>>> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
>>> >>>> upgrading Curator to 5.x, which would allow us to properly fix an
>>> >>>> issue
>>> >>>> with inconsistent state. It is also required to eventually support
>>> ZK
>>> >>> 3.6.
>>> >
>>> >
>>>
>>>
>>
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Chesnay Schepler <ch...@apache.org>.
Since this is only relevant for 1.15, if you intend to migrate to 1.15
close to the release, then somewhere around February.
The only resource I could find for migrating Zookeeper is this FAQ:
https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ
On 07/12/2021 04:02, Dongwon Kim wrote:
> When should I prepare for upgrading ZK to 3.5 or newer?
> We're operating a Hadoop cluster w/ ZK 3.4.6 for running only Flink jobs.
> Just hope that the rolling update is not that painful - any advice on
> this?
>
> Best,
>
> Dongwon
>
> On Tue, Dec 7, 2021 at 3:22 AM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> Current users of ZK 3.4 and below would need to upgrade their
> Zookeeper installation that is used by Flink to 3.5+.
>
> Whether K8s users are affected depends on whether they use ZK or
> not. If they do, see above, otherwise they are not affected at all.
>
> On 06/12/2021 18:49, Arvid Heise wrote:
>> Could someone please help me understand the implications of the
>> upgrade?
>>
>> As far as I understood this upgrade would only affect users that
>> have a zookeeper shared across multiple services, some of which
>> require ZK 3.4-? A workaround for those users would be to run two
>> ZKs with different versions, eventually deprecating old ZK, correct?
>>
>> If that is the only limitation, I'm +1 for the proposal since ZK
>> 3.4 is already EOL.
>>
>> How are K8s users affected?
>>
>> Best,
>>
>> Arvid
>>
>> On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler
>> <ch...@apache.org> wrote:
>>
>> ping @users; any input on how this would affect you is highly
>> appreciated.
>>
>> On 25/11/2021 22:39, Chesnay Schepler wrote:
>> > I included the user ML in the thread.
>> >
>> > @users Are you still using Zookeeper 3.4? If so, were you
>> planning to
>> > upgrade Zookeeper in the near future?
>> >
>> > I'm not sure about ZK compatibility, but we'd also upgrade
>> Curator to
>> > 5.x, which doesn't support ookeeperK 3.4 anymore.
>> >
>> > On 25/11/2021 21:56, Till Rohrmann wrote:
>> >> Should we ask on the user mailing list whether anybody is
>> still using
>> >> ZooKeeper 3.4 and thus needs support for this version or
>> can a ZooKeeper
>> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would
>> expect that
>> >> not a
>> >> lot of users depend on it but just to make sure that we
>> aren't
>> >> annoying a
>> >> lot of our users with this change. Apart from that +1 for
>> removing it if
>> >> not a lot of user depend on it.
>> >>
>> >> Cheers,
>> >> Till
>> >>
>> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl
>> <ma...@ververica.com>
>> >> wrote:
>> >>
>> >>> Thanks for starting this discussion, Chesnay. +1 from my
>> side. It's
>> >>> time to
>> >>> move forward with the ZK support considering the EOL of
>> 3.4 you already
>> >>> mentioned. The benefits we gain from upgrading Curator to
>> 5.x as a
>> >>> consequence is another plus point. Just for reference on the
>> >>> inconsistent
>> >>> state issue you mentioned: FLINK-24543 [1].
>> >>>
>> >>> Matthias
>> >>>
>> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
>> >>>
>> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler
>> <ch...@apache.org>
>> >>> wrote:
>> >>>
>> >>>> Hello,
>> >>>>
>> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15,
>> upgrading the
>> >>>> default to 3.5 with an opt-in for 3.6.
>> >>>>
>> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents
>> us from
>> >>>> upgrading Curator to 5.x, which would allow us to
>> properly fix an
>> >>>> issue
>> >>>> with inconsistent state. It is also required to
>> eventually support ZK
>> >>> 3.6.
>> >
>> >
>>
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Chesnay Schepler <ch...@apache.org>.
Since this is only relevant for 1.15, if you intend to migrate to 1.15
close to the release, then somewhere around February.
The only resource I could find for migrating Zookeeper is this FAQ:
https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ
On 07/12/2021 04:02, Dongwon Kim wrote:
> When should I prepare for upgrading ZK to 3.5 or newer?
> We're operating a Hadoop cluster w/ ZK 3.4.6 for running only Flink jobs.
> Just hope that the rolling update is not that painful - any advice on
> this?
>
> Best,
>
> Dongwon
>
> On Tue, Dec 7, 2021 at 3:22 AM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> Current users of ZK 3.4 and below would need to upgrade their
> Zookeeper installation that is used by Flink to 3.5+.
>
> Whether K8s users are affected depends on whether they use ZK or
> not. If they do, see above, otherwise they are not affected at all.
>
> On 06/12/2021 18:49, Arvid Heise wrote:
>> Could someone please help me understand the implications of the
>> upgrade?
>>
>> As far as I understood this upgrade would only affect users that
>> have a zookeeper shared across multiple services, some of which
>> require ZK 3.4-? A workaround for those users would be to run two
>> ZKs with different versions, eventually deprecating old ZK, correct?
>>
>> If that is the only limitation, I'm +1 for the proposal since ZK
>> 3.4 is already EOL.
>>
>> How are K8s users affected?
>>
>> Best,
>>
>> Arvid
>>
>> On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler
>> <ch...@apache.org> wrote:
>>
>> ping @users; any input on how this would affect you is highly
>> appreciated.
>>
>> On 25/11/2021 22:39, Chesnay Schepler wrote:
>> > I included the user ML in the thread.
>> >
>> > @users Are you still using Zookeeper 3.4? If so, were you
>> planning to
>> > upgrade Zookeeper in the near future?
>> >
>> > I'm not sure about ZK compatibility, but we'd also upgrade
>> Curator to
>> > 5.x, which doesn't support ookeeperK 3.4 anymore.
>> >
>> > On 25/11/2021 21:56, Till Rohrmann wrote:
>> >> Should we ask on the user mailing list whether anybody is
>> still using
>> >> ZooKeeper 3.4 and thus needs support for this version or
>> can a ZooKeeper
>> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would
>> expect that
>> >> not a
>> >> lot of users depend on it but just to make sure that we
>> aren't
>> >> annoying a
>> >> lot of our users with this change. Apart from that +1 for
>> removing it if
>> >> not a lot of user depend on it.
>> >>
>> >> Cheers,
>> >> Till
>> >>
>> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl
>> <ma...@ververica.com>
>> >> wrote:
>> >>
>> >>> Thanks for starting this discussion, Chesnay. +1 from my
>> side. It's
>> >>> time to
>> >>> move forward with the ZK support considering the EOL of
>> 3.4 you already
>> >>> mentioned. The benefits we gain from upgrading Curator to
>> 5.x as a
>> >>> consequence is another plus point. Just for reference on the
>> >>> inconsistent
>> >>> state issue you mentioned: FLINK-24543 [1].
>> >>>
>> >>> Matthias
>> >>>
>> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
>> >>>
>> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler
>> <ch...@apache.org>
>> >>> wrote:
>> >>>
>> >>>> Hello,
>> >>>>
>> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15,
>> upgrading the
>> >>>> default to 3.5 with an opt-in for 3.6.
>> >>>>
>> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents
>> us from
>> >>>> upgrading Curator to 5.x, which would allow us to
>> properly fix an
>> >>>> issue
>> >>>> with inconsistent state. It is also required to
>> eventually support ZK
>> >>> 3.6.
>> >
>> >
>>
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Dongwon Kim <ea...@gmail.com>.
When should I prepare for upgrading ZK to 3.5 or newer?
We're operating a Hadoop cluster w/ ZK 3.4.6 for running only Flink jobs.
Just hope that the rolling update is not that painful - any advice on this?
Best,
Dongwon
On Tue, Dec 7, 2021 at 3:22 AM Chesnay Schepler <ch...@apache.org> wrote:
> Current users of ZK 3.4 and below would need to upgrade their Zookeeper
> installation that is used by Flink to 3.5+.
>
> Whether K8s users are affected depends on whether they use ZK or not. If
> they do, see above, otherwise they are not affected at all.
>
> On 06/12/2021 18:49, Arvid Heise wrote:
>
> Could someone please help me understand the implications of the upgrade?
>
> As far as I understood this upgrade would only affect users that have a
> zookeeper shared across multiple services, some of which require ZK 3.4-? A
> workaround for those users would be to run two ZKs with different versions,
> eventually deprecating old ZK, correct?
>
> If that is the only limitation, I'm +1 for the proposal since ZK 3.4 is
> already EOL.
>
> How are K8s users affected?
>
> Best,
>
> Arvid
>
> On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
>> ping @users; any input on how this would affect you is highly appreciated.
>>
>> On 25/11/2021 22:39, Chesnay Schepler wrote:
>> > I included the user ML in the thread.
>> >
>> > @users Are you still using Zookeeper 3.4? If so, were you planning to
>> > upgrade Zookeeper in the near future?
>> >
>> > I'm not sure about ZK compatibility, but we'd also upgrade Curator to
>> > 5.x, which doesn't support ookeeperK 3.4 anymore.
>> >
>> > On 25/11/2021 21:56, Till Rohrmann wrote:
>> >> Should we ask on the user mailing list whether anybody is still using
>> >> ZooKeeper 3.4 and thus needs support for this version or can a
>> ZooKeeper
>> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that
>> >> not a
>> >> lot of users depend on it but just to make sure that we aren't
>> >> annoying a
>> >> lot of our users with this change. Apart from that +1 for removing it
>> if
>> >> not a lot of user depend on it.
>> >>
>> >> Cheers,
>> >> Till
>> >>
>> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl <matthias@ververica.com
>> >
>> >> wrote:
>> >>
>> >>> Thanks for starting this discussion, Chesnay. +1 from my side. It's
>> >>> time to
>> >>> move forward with the ZK support considering the EOL of 3.4 you
>> already
>> >>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
>> >>> consequence is another plus point. Just for reference on the
>> >>> inconsistent
>> >>> state issue you mentioned: FLINK-24543 [1].
>> >>>
>> >>> Matthias
>> >>>
>> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
>> >>>
>> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler <chesnay@apache.org
>> >
>> >>> wrote:
>> >>>
>> >>>> Hello,
>> >>>>
>> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
>> >>>> default to 3.5 with an opt-in for 3.6.
>> >>>>
>> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
>> >>>> upgrading Curator to 5.x, which would allow us to properly fix an
>> >>>> issue
>> >>>> with inconsistent state. It is also required to eventually support ZK
>> >>> 3.6.
>> >
>> >
>>
>>
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Dongwon Kim <ea...@gmail.com>.
When should I prepare for upgrading ZK to 3.5 or newer?
We're operating a Hadoop cluster w/ ZK 3.4.6 for running only Flink jobs.
Just hope that the rolling update is not that painful - any advice on this?
Best,
Dongwon
On Tue, Dec 7, 2021 at 3:22 AM Chesnay Schepler <ch...@apache.org> wrote:
> Current users of ZK 3.4 and below would need to upgrade their Zookeeper
> installation that is used by Flink to 3.5+.
>
> Whether K8s users are affected depends on whether they use ZK or not. If
> they do, see above, otherwise they are not affected at all.
>
> On 06/12/2021 18:49, Arvid Heise wrote:
>
> Could someone please help me understand the implications of the upgrade?
>
> As far as I understood this upgrade would only affect users that have a
> zookeeper shared across multiple services, some of which require ZK 3.4-? A
> workaround for those users would be to run two ZKs with different versions,
> eventually deprecating old ZK, correct?
>
> If that is the only limitation, I'm +1 for the proposal since ZK 3.4 is
> already EOL.
>
> How are K8s users affected?
>
> Best,
>
> Arvid
>
> On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
>> ping @users; any input on how this would affect you is highly appreciated.
>>
>> On 25/11/2021 22:39, Chesnay Schepler wrote:
>> > I included the user ML in the thread.
>> >
>> > @users Are you still using Zookeeper 3.4? If so, were you planning to
>> > upgrade Zookeeper in the near future?
>> >
>> > I'm not sure about ZK compatibility, but we'd also upgrade Curator to
>> > 5.x, which doesn't support ookeeperK 3.4 anymore.
>> >
>> > On 25/11/2021 21:56, Till Rohrmann wrote:
>> >> Should we ask on the user mailing list whether anybody is still using
>> >> ZooKeeper 3.4 and thus needs support for this version or can a
>> ZooKeeper
>> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that
>> >> not a
>> >> lot of users depend on it but just to make sure that we aren't
>> >> annoying a
>> >> lot of our users with this change. Apart from that +1 for removing it
>> if
>> >> not a lot of user depend on it.
>> >>
>> >> Cheers,
>> >> Till
>> >>
>> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl <matthias@ververica.com
>> >
>> >> wrote:
>> >>
>> >>> Thanks for starting this discussion, Chesnay. +1 from my side. It's
>> >>> time to
>> >>> move forward with the ZK support considering the EOL of 3.4 you
>> already
>> >>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
>> >>> consequence is another plus point. Just for reference on the
>> >>> inconsistent
>> >>> state issue you mentioned: FLINK-24543 [1].
>> >>>
>> >>> Matthias
>> >>>
>> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
>> >>>
>> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler <chesnay@apache.org
>> >
>> >>> wrote:
>> >>>
>> >>>> Hello,
>> >>>>
>> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
>> >>>> default to 3.5 with an opt-in for 3.6.
>> >>>>
>> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
>> >>>> upgrading Curator to 5.x, which would allow us to properly fix an
>> >>>> issue
>> >>>> with inconsistent state. It is also required to eventually support ZK
>> >>> 3.6.
>> >
>> >
>>
>>
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Yang Wang <da...@gmail.com>.
FYI:
We(Alibaba) are widely using ZooKeeper 3.5.5 for all the YARN and some K8s
Flink high-available applications.
Best,
Yang
Chesnay Schepler <ch...@apache.org> 于2021年12月7日周二 上午2:22写道：
> Current users of ZK 3.4 and below would need to upgrade their Zookeeper
> installation that is used by Flink to 3.5+.
>
> Whether K8s users are affected depends on whether they use ZK or not. If
> they do, see above, otherwise they are not affected at all.
>
> On 06/12/2021 18:49, Arvid Heise wrote:
> > Could someone please help me understand the implications of the upgrade?
> >
> > As far as I understood this upgrade would only affect users that have
> > a zookeeper shared across multiple services, some of which require ZK
> > 3.4-? A workaround for those users would be to run two ZKs with
> > different versions, eventually deprecating old ZK, correct?
> >
> > If that is the only limitation, I'm +1 for the proposal since ZK 3.4
> > is already EOL.
> >
> > How are K8s users affected?
> >
> > Best,
> >
> > Arvid
> >
> > On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org>
> > wrote:
> >
> > ping @users; any input on how this would affect you is highly
> > appreciated.
> >
> > On 25/11/2021 22:39, Chesnay Schepler wrote:
> > > I included the user ML in the thread.
> > >
> > > @users Are you still using Zookeeper 3.4? If so, were you
> > planning to
> > > upgrade Zookeeper in the near future?
> > >
> > > I'm not sure about ZK compatibility, but we'd also upgrade
> > Curator to
> > > 5.x, which doesn't support ookeeperK 3.4 anymore.
> > >
> > > On 25/11/2021 21:56, Till Rohrmann wrote:
> > >> Should we ask on the user mailing list whether anybody is still
> > using
> > >> ZooKeeper 3.4 and thus needs support for this version or can a
> > ZooKeeper
> > >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect
> > that
> > >> not a
> > >> lot of users depend on it but just to make sure that we aren't
> > >> annoying a
> > >> lot of our users with this change. Apart from that +1 for
> > removing it if
> > >> not a lot of user depend on it.
> > >>
> > >> Cheers,
> > >> Till
> > >>
> > >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl
> > <ma...@ververica.com>
> > >> wrote:
> > >>
> > >>> Thanks for starting this discussion, Chesnay. +1 from my side.
> > It's
> > >>> time to
> > >>> move forward with the ZK support considering the EOL of 3.4
> > you already
> > >>> mentioned. The benefits we gain from upgrading Curator to 5.x as
> a
> > >>> consequence is another plus point. Just for reference on the
> > >>> inconsistent
> > >>> state issue you mentioned: FLINK-24543 [1].
> > >>>
> > >>> Matthias
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
> > >>>
> > >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler
> > <ch...@apache.org>
> > >>> wrote:
> > >>>
> > >>>> Hello,
> > >>>>
> > >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading
> the
> > >>>> default to 3.5 with an opt-in for 3.6.
> > >>>>
> > >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
> > >>>> upgrading Curator to 5.x, which would allow us to properly
> > fix an
> > >>>> issue
> > >>>> with inconsistent state. It is also required to eventually
> > support ZK
> > >>> 3.6.
> > >
> > >
> >
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Yang Wang <da...@gmail.com>.
FYI:
We(Alibaba) are widely using ZooKeeper 3.5.5 for all the YARN and some K8s
Flink high-available applications.
Best,
Yang
Chesnay Schepler <ch...@apache.org> 于2021年12月7日周二 上午2:22写道：
> Current users of ZK 3.4 and below would need to upgrade their Zookeeper
> installation that is used by Flink to 3.5+.
>
> Whether K8s users are affected depends on whether they use ZK or not. If
> they do, see above, otherwise they are not affected at all.
>
> On 06/12/2021 18:49, Arvid Heise wrote:
> > Could someone please help me understand the implications of the upgrade?
> >
> > As far as I understood this upgrade would only affect users that have
> > a zookeeper shared across multiple services, some of which require ZK
> > 3.4-? A workaround for those users would be to run two ZKs with
> > different versions, eventually deprecating old ZK, correct?
> >
> > If that is the only limitation, I'm +1 for the proposal since ZK 3.4
> > is already EOL.
> >
> > How are K8s users affected?
> >
> > Best,
> >
> > Arvid
> >
> > On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org>
> > wrote:
> >
> > ping @users; any input on how this would affect you is highly
> > appreciated.
> >
> > On 25/11/2021 22:39, Chesnay Schepler wrote:
> > > I included the user ML in the thread.
> > >
> > > @users Are you still using Zookeeper 3.4? If so, were you
> > planning to
> > > upgrade Zookeeper in the near future?
> > >
> > > I'm not sure about ZK compatibility, but we'd also upgrade
> > Curator to
> > > 5.x, which doesn't support ookeeperK 3.4 anymore.
> > >
> > > On 25/11/2021 21:56, Till Rohrmann wrote:
> > >> Should we ask on the user mailing list whether anybody is still
> > using
> > >> ZooKeeper 3.4 and thus needs support for this version or can a
> > ZooKeeper
> > >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect
> > that
> > >> not a
> > >> lot of users depend on it but just to make sure that we aren't
> > >> annoying a
> > >> lot of our users with this change. Apart from that +1 for
> > removing it if
> > >> not a lot of user depend on it.
> > >>
> > >> Cheers,
> > >> Till
> > >>
> > >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl
> > <ma...@ververica.com>
> > >> wrote:
> > >>
> > >>> Thanks for starting this discussion, Chesnay. +1 from my side.
> > It's
> > >>> time to
> > >>> move forward with the ZK support considering the EOL of 3.4
> > you already
> > >>> mentioned. The benefits we gain from upgrading Curator to 5.x as
> a
> > >>> consequence is another plus point. Just for reference on the
> > >>> inconsistent
> > >>> state issue you mentioned: FLINK-24543 [1].
> > >>>
> > >>> Matthias
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
> > >>>
> > >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler
> > <ch...@apache.org>
> > >>> wrote:
> > >>>
> > >>>> Hello,
> > >>>>
> > >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading
> the
> > >>>> default to 3.5 with an opt-in for 3.6.
> > >>>>
> > >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
> > >>>> upgrading Curator to 5.x, which would allow us to properly
> > fix an
> > >>>> issue
> > >>>> with inconsistent state. It is also required to eventually
> > support ZK
> > >>> 3.6.
> > >
> > >
> >
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Chesnay Schepler <ch...@apache.org>.
Current users of ZK 3.4 and below would need to upgrade their Zookeeper
installation that is used by Flink to 3.5+.
Whether K8s users are affected depends on whether they use ZK or not. If
they do, see above, otherwise they are not affected at all.
On 06/12/2021 18:49, Arvid Heise wrote:
> Could someone please help me understand the implications of the upgrade?
>
> As far as I understood this upgrade would only affect users that have
> a zookeeper shared across multiple services, some of which require ZK
> 3.4-? A workaround for those users would be to run two ZKs with
> different versions, eventually deprecating old ZK, correct?
>
> If that is the only limitation, I'm +1 for the proposal since ZK 3.4
> is already EOL.
>
> How are K8s users affected?
>
> Best,
>
> Arvid
>
> On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> ping @users; any input on how this would affect you is highly
> appreciated.
>
> On 25/11/2021 22:39, Chesnay Schepler wrote:
> > I included the user ML in the thread.
> >
> > @users Are you still using Zookeeper 3.4? If so, were you
> planning to
> > upgrade Zookeeper in the near future?
> >
> > I'm not sure about ZK compatibility, but we'd also upgrade
> Curator to
> > 5.x, which doesn't support ookeeperK 3.4 anymore.
> >
> > On 25/11/2021 21:56, Till Rohrmann wrote:
> >> Should we ask on the user mailing list whether anybody is still
> using
> >> ZooKeeper 3.4 and thus needs support for this version or can a
> ZooKeeper
> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect
> that
> >> not a
> >> lot of users depend on it but just to make sure that we aren't
> >> annoying a
> >> lot of our users with this change. Apart from that +1 for
> removing it if
> >> not a lot of user depend on it.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl
> <ma...@ververica.com>
> >> wrote:
> >>
> >>> Thanks for starting this discussion, Chesnay. +1 from my side.
> It's
> >>> time to
> >>> move forward with the ZK support considering the EOL of 3.4
> you already
> >>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
> >>> consequence is another plus point. Just for reference on the
> >>> inconsistent
> >>> state issue you mentioned: FLINK-24543 [1].
> >>>
> >>> Matthias
> >>>
> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
> >>>
> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler
> <ch...@apache.org>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
> >>>> default to 3.5 with an opt-in for 3.6.
> >>>>
> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
> >>>> upgrading Curator to 5.x, which would allow us to properly
> fix an
> >>>> issue
> >>>> with inconsistent state. It is also required to eventually
> support ZK
> >>> 3.6.
> >
> >
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Chesnay Schepler <ch...@apache.org>.
Current users of ZK 3.4 and below would need to upgrade their Zookeeper
installation that is used by Flink to 3.5+.
Whether K8s users are affected depends on whether they use ZK or not. If
they do, see above, otherwise they are not affected at all.
On 06/12/2021 18:49, Arvid Heise wrote:
> Could someone please help me understand the implications of the upgrade?
>
> As far as I understood this upgrade would only affect users that have
> a zookeeper shared across multiple services, some of which require ZK
> 3.4-? A workaround for those users would be to run two ZKs with
> different versions, eventually deprecating old ZK, correct?
>
> If that is the only limitation, I'm +1 for the proposal since ZK 3.4
> is already EOL.
>
> How are K8s users affected?
>
> Best,
>
> Arvid
>
> On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> ping @users; any input on how this would affect you is highly
> appreciated.
>
> On 25/11/2021 22:39, Chesnay Schepler wrote:
> > I included the user ML in the thread.
> >
> > @users Are you still using Zookeeper 3.4? If so, were you
> planning to
> > upgrade Zookeeper in the near future?
> >
> > I'm not sure about ZK compatibility, but we'd also upgrade
> Curator to
> > 5.x, which doesn't support ookeeperK 3.4 anymore.
> >
> > On 25/11/2021 21:56, Till Rohrmann wrote:
> >> Should we ask on the user mailing list whether anybody is still
> using
> >> ZooKeeper 3.4 and thus needs support for this version or can a
> ZooKeeper
> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect
> that
> >> not a
> >> lot of users depend on it but just to make sure that we aren't
> >> annoying a
> >> lot of our users with this change. Apart from that +1 for
> removing it if
> >> not a lot of user depend on it.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl
> <ma...@ververica.com>
> >> wrote:
> >>
> >>> Thanks for starting this discussion, Chesnay. +1 from my side.
> It's
> >>> time to
> >>> move forward with the ZK support considering the EOL of 3.4
> you already
> >>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
> >>> consequence is another plus point. Just for reference on the
> >>> inconsistent
> >>> state issue you mentioned: FLINK-24543 [1].
> >>>
> >>> Matthias
> >>>
> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
> >>>
> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler
> <ch...@apache.org>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
> >>>> default to 3.5 with an opt-in for 3.6.
> >>>>
> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
> >>>> upgrading Curator to 5.x, which would allow us to properly
> fix an
> >>>> issue
> >>>> with inconsistent state. It is also required to eventually
> support ZK
> >>> 3.6.
> >
> >
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Arvid Heise <ar...@apache.org>.
Could someone please help me understand the implications of the upgrade?
As far as I understood this upgrade would only affect users that have a
zookeeper shared across multiple services, some of which require ZK 3.4-? A
workaround for those users would be to run two ZKs with different versions,
eventually deprecating old ZK, correct?
If that is the only limitation, I'm +1 for the proposal since ZK 3.4 is
already EOL.
How are K8s users affected?
Best,
Arvid
On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org> wrote:
> ping @users; any input on how this would affect you is highly appreciated.
>
> On 25/11/2021 22:39, Chesnay Schepler wrote:
> > I included the user ML in the thread.
> >
> > @users Are you still using Zookeeper 3.4? If so, were you planning to
> > upgrade Zookeeper in the near future?
> >
> > I'm not sure about ZK compatibility, but we'd also upgrade Curator to
> > 5.x, which doesn't support ookeeperK 3.4 anymore.
> >
> > On 25/11/2021 21:56, Till Rohrmann wrote:
> >> Should we ask on the user mailing list whether anybody is still using
> >> ZooKeeper 3.4 and thus needs support for this version or can a ZooKeeper
> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that
> >> not a
> >> lot of users depend on it but just to make sure that we aren't
> >> annoying a
> >> lot of our users with this change. Apart from that +1 for removing it if
> >> not a lot of user depend on it.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl <ma...@ververica.com>
> >> wrote:
> >>
> >>> Thanks for starting this discussion, Chesnay. +1 from my side. It's
> >>> time to
> >>> move forward with the ZK support considering the EOL of 3.4 you already
> >>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
> >>> consequence is another plus point. Just for reference on the
> >>> inconsistent
> >>> state issue you mentioned: FLINK-24543 [1].
> >>>
> >>> Matthias
> >>>
> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
> >>>
> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler <ch...@apache.org>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
> >>>> default to 3.5 with an opt-in for 3.6.
> >>>>
> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
> >>>> upgrading Curator to 5.x, which would allow us to properly fix an
> >>>> issue
> >>>> with inconsistent state. It is also required to eventually support ZK
> >>> 3.6.
> >
> >
>
>
Re: [DISCUSS] Drop Zookeeper 3.4
Posted by Arvid Heise <ar...@apache.org>.
Could someone please help me understand the implications of the upgrade?
As far as I understood this upgrade would only affect users that have a
zookeeper shared across multiple services, some of which require ZK 3.4-? A
workaround for those users would be to run two ZKs with different versions,
eventually deprecating old ZK, correct?
If that is the only limitation, I'm +1 for the proposal since ZK 3.4 is
already EOL.
How are K8s users affected?
Best,
Arvid
On Mon, Dec 6, 2021 at 2:00 PM Chesnay Schepler <ch...@apache.org> wrote:
> ping @users; any input on how this would affect you is highly appreciated.
>
> On 25/11/2021 22:39, Chesnay Schepler wrote:
> > I included the user ML in the thread.
> >
> > @users Are you still using Zookeeper 3.4? If so, were you planning to
> > upgrade Zookeeper in the near future?
> >
> > I'm not sure about ZK compatibility, but we'd also upgrade Curator to
> > 5.x, which doesn't support ookeeperK 3.4 anymore.
> >
> > On 25/11/2021 21:56, Till Rohrmann wrote:
> >> Should we ask on the user mailing list whether anybody is still using
> >> ZooKeeper 3.4 and thus needs support for this version or can a ZooKeeper
> >> 3.5/3.6 client talk to a ZooKeeper 3.4 cluster? I would expect that
> >> not a
> >> lot of users depend on it but just to make sure that we aren't
> >> annoying a
> >> lot of our users with this change. Apart from that +1 for removing it if
> >> not a lot of user depend on it.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Wed, Nov 24, 2021 at 11:03 AM Matthias Pohl <ma...@ververica.com>
> >> wrote:
> >>
> >>> Thanks for starting this discussion, Chesnay. +1 from my side. It's
> >>> time to
> >>> move forward with the ZK support considering the EOL of 3.4 you already
> >>> mentioned. The benefits we gain from upgrading Curator to 5.x as a
> >>> consequence is another plus point. Just for reference on the
> >>> inconsistent
> >>> state issue you mentioned: FLINK-24543 [1].
> >>>
> >>> Matthias
> >>>
> >>> [1] https://issues.apache.org/jira/browse/FLINK-24543
> >>>
> >>> On Wed, Nov 24, 2021 at 10:19 AM Chesnay Schepler <ch...@apache.org>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> I'd like to drop support for Zookeeper 3.4 in 1.15, upgrading the
> >>>> default to 3.5 with an opt-in for 3.6.
> >>>>
> >>>> Supporting Zookeeper 3.4 (which is already EOL) prevents us from
> >>>> upgrading Curator to 5.x, which would allow us to properly fix an
> >>>> issue
> >>>> with inconsistent state. It is also required to eventually support ZK
> >>> 3.6.
> >
> >
>
>