Why are errors stored in the transaction logs?

[Sylvain Wallez <sy...@apache.org>]

Hi ZooKeepers,

We had an issue recently with a service hammering ZK to read a path that 
doesn't exist. As a result the transaction logs grew quickly because 
errors are stored there.

Looking at DataTree.processTxn() which is called from 
FileTxnSnapLog.restore() it seems that errors read in the transaction 
log are essentially ignored.

So what is storing errors in the txnlog useful for? Would it make sense 
to have a configuration flag to not store them?

Thanks,
Sylvain

Re: Why are errors stored in the transaction logs?

[Andor Molnar <an...@apache.org>]

Hi Sylvain,

That should not be the case. Txns are not going to be created from read requests and not hit SyncRequestProcessor which is responsible for maintaining txn and snap logs.

The error handling you see in processTxn() is for ignoring failures during log replaying inside processTxn() method (in the catch branch). Introduced in ZOOKEEPER-1269 and it’s related to multis afaics.

Andor




> On 2019. Oct 15., at 16:33, Sylvain Wallez <sy...@apache.org> wrote:
> 
> Hi ZooKeepers,
> 
> We had an issue recently with a service hammering ZK to read a path that doesn't exist. As a result the transaction logs grew quickly because errors are stored there.
> 
> Looking at DataTree.processTxn() which is called from FileTxnSnapLog.restore() it seems that errors read in the transaction log are essentially ignored.
> 
> So what is storing errors in the txnlog useful for? Would it make sense to have a configuration flag to not store them?
> 
> Thanks,
> Sylvain
>