You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Li Wang <li...@gmail.com> on 2021/03/01 21:00:09 UTC
Re: quota stats data consistency
Some updates on this.
I looked more into this and found that quota usage is actually
re-calculated when deserializing the data tree from the snapshot, so the
quota data will be consistent even in the case zk server crashes.
Best,
Li
On Tue, Feb 23, 2021 at 10:13 AM Li Wang <li...@gmail.com> wrote:
>
> Hi,
>
> The Quota stats are updated by calling updateQuotaStat() API in the
> FinalRequestProcessor. However, the updateQuotaStat() API only updates the
> in-memory data tree.
>
> In the scenario that a ZK server crashed right before the updateQuotaStat
> is called or a snapshot was taken, are we going to have the following data
> inconsistency issue?
>
> 1. the quota stats is inaccurate in the crashed server as the
> write transaction has been logged by the SyncRequestProcessor but the
> quota stats update was lost?
>
> 2. the quota stats will be inconsistent across different servers as the
> quota stats persisted in other servers via snapshot but not the crashed one
>
>
> updateQuotaStat() code snippet
> ========================
> synchronized (statNode) {
> updatedStat = new StatsTrack(statNode.data);
> updatedStat.setCount(updatedStat.getCount() + countDiff);
> updatedStat.setBytes(updatedStat.getBytes() + bytesDiff);
>
> statNode.data = updatedStat.getStatsBytes();
> }
>
> Any thoughts?
>
> Thanks,
>
> Li
>
Re: quota stats data consistency
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
this is a great news!
Thanks for raising the issue and also for digging into it!
Regards,
Mate
On Mon, Mar 1, 2021 at 10:00 PM Li Wang <li...@gmail.com> wrote:
> Some updates on this.
>
> I looked more into this and found that quota usage is actually
> re-calculated when deserializing the data tree from the snapshot, so the
> quota data will be consistent even in the case zk server crashes.
>
> Best,
>
> Li
>
> On Tue, Feb 23, 2021 at 10:13 AM Li Wang <li...@gmail.com> wrote:
>
> >
> > Hi,
> >
> > The Quota stats are updated by calling updateQuotaStat() API in the
> > FinalRequestProcessor. However, the updateQuotaStat() API only updates
> the
> > in-memory data tree.
> >
> > In the scenario that a ZK server crashed right before the updateQuotaStat
> > is called or a snapshot was taken, are we going to have the following
> data
> > inconsistency issue?
> >
> > 1. the quota stats is inaccurate in the crashed server as the
> > write transaction has been logged by the SyncRequestProcessor but the
> > quota stats update was lost?
> >
> > 2. the quota stats will be inconsistent across different servers as the
> > quota stats persisted in other servers via snapshot but not the crashed
> one
> >
> >
> > updateQuotaStat() code snippet
> > ========================
> > synchronized (statNode) {
> > updatedStat = new StatsTrack(statNode.data);
> > updatedStat.setCount(updatedStat.getCount() + countDiff);
> > updatedStat.setBytes(updatedStat.getBytes() + bytesDiff);
> >
> > statNode.data = updatedStat.getStatsBytes();
> > }
> >
> > Any thoughts?
> >
> > Thanks,
> >
> > Li
> >
>