You are viewing a plain text version of this content. The canonical link for it is here.
Posted to distributedlog-user@bookkeeper.apache.org by "Veloso, Vasco (Coriant - PT/Lisbon)" <Va...@coriant.com> on 2017/12/21 09:38:45 UTC

Recurring "directory not empty" errors on zookeeper log

Hello,

I'm a bit of a newbie on things regarding Bookkeeper, Zookeeper and DistributedLog so sorry if this seems obvious. I thought I had understood this ecosystem but obviously I still have a long way to go. :)

I'm evaluating the DistributedLog package for a project and among other things I'm scratching my head at a line that pops up regularly in my Zookeeper logs:

2017-12-21 09:18:04,768 - INFO  - [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@880] - Got user-level KeeperException when processing sessionid:0x10072c2295d000c type:delete cxid:0x147 zxid:0x251 txntype:-1 reqpath:n/a Error Path:/messaging/bookkeeper/ledgers/00/0000 Error:KeeperErrorCode = Directory not empty for /messaging/bookkeeper/ledgers/00/0000

It looks like metadata management is leaving stuff behind in Zookeeper.

My gut feeling is that something is wrong. Is it really?

I'm using the Bookkeeper and Zookeeper jars distributed with the 0.5.0 release of DistributedLog, if that matters.

Thanks,

Vasco

Re: Recurring "directory not empty" errors on zookeeper log

Posted by "Veloso, Vasco (Coriant - PT/Lisbon)" <Va...@coriant.com>.
Hi,

Sure, that makes sense.

Thanks for the explanation.

Vasco
Em 21/12/2017, em 17:47, Sijie Guo <gu...@gmail.com>> escreveu:


On Thu, Dec 21, 2017 at 1:38 AM, Veloso, Vasco (Coriant - PT/Lisbon) <Va...@coriant.com>> wrote:

Hello,



I’m a bit of a newbie on things regarding Bookkeeper, Zookeeper and DistributedLog so sorry if this seems obvious. I thought I had understood this ecosystem but obviously I still have a long way to go. ☺



I’m evaluating the DistributedLog package for a project and among other things I’m scratching my head at a line that pops up regularly in my Zookeeper logs:



2017-12-21 09:18:04,768 - INFO  - [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@880] - Got user-level KeeperException when processing sessionid:0x10072c2295d000c type:delete cxid:0x147 zxid:0x251 txntype:-1 reqpath:n/a Error Path:/messaging/bookkeeper/ledgers/00/0000 Error:KeeperErrorCode = Directory not empty for /messaging/bookkeeper/ledgers/00/0000



It looks like metadata management is leaving stuff behind in Zookeeper.



My gut feeling is that something is wrong. Is it really?

Oh, I don't think it is a problem here. /messaging/bookkeeper/ ledgers/00/0000 is the path used by bookkeeper for storing metadata for ledgers.
There is a logic in bookkeeper ledger manager: when a ledger is deleted, it will delete its metadata znode from zookeeper, after that it will attempt
to delete its parent znode to clean up. If the parent znode has other znodes, it can't be deleted. that's what you see in the log. It is a verbose logging
from zookeeper, a bit annoying. You can turn the logging level of zookeeper class to warning by setting `log4j.logger.org.apache.zookeeper` to `WARN`
in the log4j properties file.

I also created a bookkeeper issue to improve the logic there, so it won't have such logging from zookeeper.  https://github.com/apache/bookkeeper/issues/902

Let me know if this makes sense to you or not.



I’m using the Bookkeeper and Zookeeper jars distributed with the 0.5.0 release of DistributedLog, if that matters.



Thanks,



Vasco

Re: Recurring "directory not empty" errors on zookeeper log

Posted by Sijie Guo <gu...@gmail.com>.
On Thu, Dec 21, 2017 at 1:38 AM, Veloso, Vasco (Coriant - PT/Lisbon) <
Vasco.Veloso@coriant.com> wrote:

> Hello,
>
>
>
> I’m a bit of a newbie on things regarding Bookkeeper, Zookeeper and
> DistributedLog so sorry if this seems obvious. I thought I had understood
> this ecosystem but obviously I still have a long way to go. J
>
>
>
> I’m evaluating the DistributedLog package for a project and among other
> things I’m scratching my head at a line that pops up regularly in my
> Zookeeper logs:
>
>
>
> 2017-12-21 09:18:04,768 - INFO  - [ProcessThread(sid:0 cport:2181)::
> PrepRequestProcessor@880] - Got user-level KeeperException when
> processing sessionid:0x10072c2295d000c type:delete cxid:0x147 zxid:0x251
> txntype:-1 reqpath:n/a Error Path:/messaging/bookkeeper/ledgers/00/0000
> Error:KeeperErrorCode = Directory not empty for
> /messaging/bookkeeper/ledgers/00/0000
>

>
> It looks like metadata management is leaving stuff behind in Zookeeper.
>
>
>
> My gut feeling is that something is wrong. Is it really?
>

Oh, I don't think it is a problem here. /messaging/bookkeeper/ledgers/00/0000
is the path used by bookkeeper for storing metadata for ledgers.
There is a logic in bookkeeper ledger manager: when a ledger is deleted, it
will delete its metadata znode from zookeeper, after that it will attempt
to delete its parent znode to clean up. If the parent znode has other
znodes, it can't be deleted. that's what you see in the log. It is a
verbose logging
from zookeeper, a bit annoying. You can turn the logging level of zookeeper
class to warning by setting `log4j.logger.org.apache.zookeeper` to `WARN`
in the log4j properties file.

I also created a bookkeeper issue to improve the logic there, so it won't
have such logging from zookeeper.
https://github.com/apache/bookkeeper/issues/902

Let me know if this makes sense to you or not.

>
>
> I’m using the Bookkeeper and Zookeeper jars distributed with the 0.5.0
> release of DistributedLog, if that matters.
>
>
>
> Thanks,
>
>
>
> Vasco
>
>
>