You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by 殷芳玺 <yi...@kuaishou.com> on 2019/10/22 10:39:20 UTC

It is necessary to backport ZOOKEEPER-3104 to branch-3.4

Hi, Folks:


ZOOKEEPER-3104 is a critical issue for data inconsistency. The risk also exists in 3.4 branch.

In one of our 3.4.13 clusters, the data inconsistency happens for many times.

After digging some transaction logs and snapshot, we believe that ZOOKEEPER-3104<https://issues.apache.org/jira/browse/ZOOKEEPER-3104> is the main risk to contributes to our data inconsistency.

The risk probability maybe higher than we can consider in real product environment.  The serialization of big DataTree may leads to a big risk time window in the high writing traffic situation. Any failure during the risk time window would cause the data inconsistency.

The data inconsistency is almost unacceptable in zookeeper semantic.

This issue is already fixed in 3.6. But I think it is very necessary to backport ZOOKEEPER-3104<https://issues.apache.org/jira/browse/ZOOKEEPER-3104> to branch-3.4, especially in the situation that the migration from 3.4 to 3.5 actually take more effort to evaluate the compatibility risk in real product environment.

I have already opened an issue [ZOOKEEPER-3589](https://issues.apache.org/jira/browse/ZOOKEEPER-3589) and submit a github pull request https://github.com/apache/zookeeper/pull/1123 to fix it.

So I suggest to accept the pull request and release it in 3.4.16. This fix would make branch-3.4 more robust and fully-fledged.


Thanks!


Best regards

Pierre Yin

答复: It is necessary to backport ZOOKEEPER-3104 to branch-3.4

Posted by 殷芳玺 <yi...@kuaishou.com>.
Hi, Enrico


Great !

Expect any feedback !


Thanks


Best Regards

Pierre

________________________________
发件人: Enrico Olivelli <eo...@gmail.com>
发送时间: 2019年10月23日 下午 1:49:46
收件人: dev@zookeeper.apache.org
主题: Re: It is necessary to backport ZOOKEEPER-3104 to branch-3.4

Il mer 23 ott 2019, 04:53 殷芳玺 <yi...@kuaishou.com> ha scritto:

> Hi，Norbert
>
>
> Great !
>
> !n my personal opinion, ZOOKEEPER-3104 backport should be necessary before
> 3.4 EOL.
>
> Zookeeper 3.4 is widely used in industry.  Any known critical issues
> should be fixed before its EOL.
>

Sure

>
> Data consistency is crucial for zookeeper.
>

Sure

>
>
> Code review comments are welcome for the pull request. Hope it be refined
> and merged to branch-3.4 upstream sooner.
>
>
> https://github.com/apache/zookeeper/pull/1123


Thank you for working on it
I am reviewing

Enrico

>
>
>
> Best Regards
>
> Pierre
>
>
>
>
> ________________________________
> 发件人: Norbert Kalmar <nk...@cloudera.com.INVALID>
> 发送时间: 2019年10月22日 下午 10:38:39
> 收件人: dev@zookeeper.apache.org
> 主题: Re: It is necessary to backport ZOOKEEPER-3104 to branch-3.4
>
> Hi Pierre,
>
> As this is a critical bugfix IMO, I don't see why it shouldn't be
> backported to 3.4. It hasn't been stated AFAIK that 3.4 is no longer
> supported.
> As for when there could be a new 3.4 release, that's a different question
> though. Let's see what the PMCs think about it.
>
> (I also added 3.5.5 to the jira's fix version, az is contains the fix but
> was missing.)
>
> Regards,
> Norbert
>
> On Tue, Oct 22, 2019 at 2:33 PM 殷芳玺 <yi...@kuaishou.com> wrote:
>
> > Hi, Folks:
> >
> >
> > ZOOKEEPER-3104 is a critical issue for data inconsistency. The risk also
> > exists in 3.4 branch.
> >
> > In one of our 3.4.13 clusters, the data inconsistency happens for many
> > times.
> >
> > After digging some transaction logs and snapshot, we believe that
> > ZOOKEEPER-3104<https://issues.apache.org/jira/browse/ZOOKEEPER-3104> is
> > the main risk to contributes to our data inconsistency.
> >
> > The risk probability maybe higher than we can consider in real product
> > environment.  The serialization of big DataTree may leads to a big risk
> > time window in the high writing traffic situation. Any failure during the
> > risk time window would cause the data inconsistency.
> >
> > The data inconsistency is almost unacceptable in zookeeper semantic.
> >
> > This issue is already fixed in 3.6. But I think it is very necessary to
> > backport ZOOKEEPER-3104<
> > https://issues.apache.org/jira/browse/ZOOKEEPER-3104> to branch-3.4,
> > especially in the situation that the migration from 3.4 to 3.5 actually
> > take more effort to evaluate the compatibility risk in real product
> > environment.
> >
> > I have already opened an issue [ZOOKEEPER-3589](
> > https://issues.apache.org/jira/browse/ZOOKEEPER-3589) and submit a
> github
> > pull request https://github.com/apache/zookeeper/pull/1123 to fix it.
> >
> > So I suggest to accept the pull request and release it in 3.4.16. This
> fix
> > would make branch-3.4 more robust and fully-fledged.
> >
> >
> > Thanks!
> >
> >
> > Best regards
> >
> > Pierre Yin
> >
> >
> >
>

Re: It is necessary to backport ZOOKEEPER-3104 to branch-3.4

Posted by Enrico Olivelli <eo...@gmail.com>.
Il mer 23 ott 2019, 04:53 殷芳玺 <yi...@kuaishou.com> ha scritto:

> Hi，Norbert
>
>
> Great !
>
> !n my personal opinion, ZOOKEEPER-3104 backport should be necessary before
> 3.4 EOL.
>
> Zookeeper 3.4 is widely used in industry.  Any known critical issues
> should be fixed before its EOL.
>

Sure

>
> Data consistency is crucial for zookeeper.
>

Sure

>
>
> Code review comments are welcome for the pull request. Hope it be refined
> and merged to branch-3.4 upstream sooner.
>
>
> https://github.com/apache/zookeeper/pull/1123


Thank you for working on it
I am reviewing

Enrico

>
>
>
> Best Regards
>
> Pierre
>
>
>
>
> ________________________________
> 发件人: Norbert Kalmar <nk...@cloudera.com.INVALID>
> 发送时间: 2019年10月22日 下午 10:38:39
> 收件人: dev@zookeeper.apache.org
> 主题: Re: It is necessary to backport ZOOKEEPER-3104 to branch-3.4
>
> Hi Pierre,
>
> As this is a critical bugfix IMO, I don't see why it shouldn't be
> backported to 3.4. It hasn't been stated AFAIK that 3.4 is no longer
> supported.
> As for when there could be a new 3.4 release, that's a different question
> though. Let's see what the PMCs think about it.
>
> (I also added 3.5.5 to the jira's fix version, az is contains the fix but
> was missing.)
>
> Regards,
> Norbert
>
> On Tue, Oct 22, 2019 at 2:33 PM 殷芳玺 <yi...@kuaishou.com> wrote:
>
> > Hi, Folks:
> >
> >
> > ZOOKEEPER-3104 is a critical issue for data inconsistency. The risk also
> > exists in 3.4 branch.
> >
> > In one of our 3.4.13 clusters, the data inconsistency happens for many
> > times.
> >
> > After digging some transaction logs and snapshot, we believe that
> > ZOOKEEPER-3104<https://issues.apache.org/jira/browse/ZOOKEEPER-3104> is
> > the main risk to contributes to our data inconsistency.
> >
> > The risk probability maybe higher than we can consider in real product
> > environment.  The serialization of big DataTree may leads to a big risk
> > time window in the high writing traffic situation. Any failure during the
> > risk time window would cause the data inconsistency.
> >
> > The data inconsistency is almost unacceptable in zookeeper semantic.
> >
> > This issue is already fixed in 3.6. But I think it is very necessary to
> > backport ZOOKEEPER-3104<
> > https://issues.apache.org/jira/browse/ZOOKEEPER-3104> to branch-3.4,
> > especially in the situation that the migration from 3.4 to 3.5 actually
> > take more effort to evaluate the compatibility risk in real product
> > environment.
> >
> > I have already opened an issue [ZOOKEEPER-3589](
> > https://issues.apache.org/jira/browse/ZOOKEEPER-3589) and submit a
> github
> > pull request https://github.com/apache/zookeeper/pull/1123 to fix it.
> >
> > So I suggest to accept the pull request and release it in 3.4.16. This
> fix
> > would make branch-3.4 more robust and fully-fledged.
> >
> >
> > Thanks!
> >
> >
> > Best regards
> >
> > Pierre Yin
> >
> >
> >
>

答复: It is necessary to backport ZOOKEEPER-3104 to branch-3.4

Posted by 殷芳玺 <yi...@kuaishou.com>.
Hi，Norbert


Great !

!n my personal opinion, ZOOKEEPER-3104 backport should be necessary before 3.4 EOL.

Zookeeper 3.4 is widely used in industry.  Any known critical issues should be fixed before its EOL.

Data consistency is crucial for zookeeper.


Code review comments are welcome for the pull request. Hope it be refined and merged to branch-3.4 upstream sooner.


https://github.com/apache/zookeeper/pull/1123


Best Regards

Pierre




________________________________
发件人: Norbert Kalmar <nk...@cloudera.com.INVALID>
发送时间: 2019年10月22日 下午 10:38:39
收件人: dev@zookeeper.apache.org
主题: Re: It is necessary to backport ZOOKEEPER-3104 to branch-3.4

Hi Pierre,

As this is a critical bugfix IMO, I don't see why it shouldn't be
backported to 3.4. It hasn't been stated AFAIK that 3.4 is no longer
supported.
As for when there could be a new 3.4 release, that's a different question
though. Let's see what the PMCs think about it.

(I also added 3.5.5 to the jira's fix version, az is contains the fix but
was missing.)

Regards,
Norbert

On Tue, Oct 22, 2019 at 2:33 PM 殷芳玺 <yi...@kuaishou.com> wrote:

> Hi, Folks:
>
>
> ZOOKEEPER-3104 is a critical issue for data inconsistency. The risk also
> exists in 3.4 branch.
>
> In one of our 3.4.13 clusters, the data inconsistency happens for many
> times.
>
> After digging some transaction logs and snapshot, we believe that
> ZOOKEEPER-3104<https://issues.apache.org/jira/browse/ZOOKEEPER-3104> is
> the main risk to contributes to our data inconsistency.
>
> The risk probability maybe higher than we can consider in real product
> environment.  The serialization of big DataTree may leads to a big risk
> time window in the high writing traffic situation. Any failure during the
> risk time window would cause the data inconsistency.
>
> The data inconsistency is almost unacceptable in zookeeper semantic.
>
> This issue is already fixed in 3.6. But I think it is very necessary to
> backport ZOOKEEPER-3104<
> https://issues.apache.org/jira/browse/ZOOKEEPER-3104> to branch-3.4,
> especially in the situation that the migration from 3.4 to 3.5 actually
> take more effort to evaluate the compatibility risk in real product
> environment.
>
> I have already opened an issue [ZOOKEEPER-3589](
> https://issues.apache.org/jira/browse/ZOOKEEPER-3589) and submit a github
> pull request https://github.com/apache/zookeeper/pull/1123 to fix it.
>
> So I suggest to accept the pull request and release it in 3.4.16. This fix
> would make branch-3.4 more robust and fully-fledged.
>
>
> Thanks!
>
>
> Best regards
>
> Pierre Yin
>
>
>

Re: It is necessary to backport ZOOKEEPER-3104 to branch-3.4

Posted by Norbert Kalmar <nk...@cloudera.com.INVALID>.
Hi Pierre,

As this is a critical bugfix IMO, I don't see why it shouldn't be
backported to 3.4. It hasn't been stated AFAIK that 3.4 is no longer
supported.
As for when there could be a new 3.4 release, that's a different question
though. Let's see what the PMCs think about it.

(I also added 3.5.5 to the jira's fix version, az is contains the fix but
was missing.)

Regards,
Norbert

On Tue, Oct 22, 2019 at 2:33 PM 殷芳玺 <yi...@kuaishou.com> wrote:

> Hi, Folks:
>
>
> ZOOKEEPER-3104 is a critical issue for data inconsistency. The risk also
> exists in 3.4 branch.
>
> In one of our 3.4.13 clusters, the data inconsistency happens for many
> times.
>
> After digging some transaction logs and snapshot, we believe that
> ZOOKEEPER-3104<https://issues.apache.org/jira/browse/ZOOKEEPER-3104> is
> the main risk to contributes to our data inconsistency.
>
> The risk probability maybe higher than we can consider in real product
> environment.  The serialization of big DataTree may leads to a big risk
> time window in the high writing traffic situation. Any failure during the
> risk time window would cause the data inconsistency.
>
> The data inconsistency is almost unacceptable in zookeeper semantic.
>
> This issue is already fixed in 3.6. But I think it is very necessary to
> backport ZOOKEEPER-3104<
> https://issues.apache.org/jira/browse/ZOOKEEPER-3104> to branch-3.4,
> especially in the situation that the migration from 3.4 to 3.5 actually
> take more effort to evaluate the compatibility risk in real product
> environment.
>
> I have already opened an issue [ZOOKEEPER-3589](
> https://issues.apache.org/jira/browse/ZOOKEEPER-3589) and submit a github
> pull request https://github.com/apache/zookeeper/pull/1123 to fix it.
>
> So I suggest to accept the pull request and release it in 3.4.16. This fix
> would make branch-3.4 more robust and fully-fledged.
>
>
> Thanks!
>
>
> Best regards
>
> Pierre Yin
>
>
>