Lock recipes and the lock path

[Jordan Zimmerman <jz...@netflix.com>]

When writing the lock recipes for ZK, there is the potential for path
garbage to build up. For example, imagine you are writing a lock for some
kind go member ID. You'd have a base path for the locks (say
"/locks/members") and then append the member ID to get locks on it. Thus,
you'd end up with "/locks/members" building up paths, one for each member
ID. 

How can you safely clean up the path for each member ID lock? There
doesn't seem to be an atomic way to do it. If one process is cleaning up
for example "/locks/members/12345", it can't be sure than another process
might need to get a lock on that member ID. I.e. There's never a safe
point to delete "/locals/members/12345".

-JZ

Re: Lock recipes and the lock path

[Ted Dunning <te...@gmail.com>]

Multi ops might help here although there might be some difficult because all
we have is an existence check and not a non-existence check.

One way to hack that is to have each normal file operation be a multi that
looks like this:

    create lock, modify file, delete lock

The lock will never be visible since it is created and deleted in a single
atomic operation.  This does guarantee that the modification will never
succeed if the lock file already exists.

The cleanup can proceed like this:

     create lock
     get children
     delete old files with large multi-op
     delete lock

This should be pretty darned fast since only for round-trips are required.

On Tue, Sep 20, 2011 at 12:37 PM, Jordan Zimmerman
<jz...@netflix.com>wrote:

> How can you safely clean up the path for each member ID lock? There
> doesn't seem to be an atomic way to do it. If one process is cleaning up
> for example "/locks/members/12345", it can't be sure than another process
> might need to get a lock on that member ID. I.e. There's never a safe
> point to delete "/locals/members/12345".
>

Re: Lock recipes and the lock path

[Jordan Zimmerman <jz...@netflix.com>]

I gave it a vote - thanks.

On 9/20/11 12:44 PM, "Fournier, Camille F." <Ca...@gs.com>
wrote:

>Well, if you are locking member IDS that are generated strings that will
>never be reused, the best you can do right now is clean those up after a
>period of time.
>
>If, on the other hand, your member IDs are meaningful, it's likely you
>will want to reuse these locations in the future when you need to get
>another lock on this member. The overhead for a node in ZK is very very
>low, so unless you are generating a ton of these nodes that you then
>never use again and never clean up, it's not likely to be too much of a
>problem.
>
>We have discussed allowing "cleanup" type paths that will delete
>themselves when there are no child nodes. See:
>https://issues.apache.org/jira/browse/ZOOKEEPER-723
>
>C

RE: Lock recipes and the lock path

["Fournier, Camille F." <Ca...@gs.com>]

Well, if you are locking member IDS that are generated strings that will never be reused, the best you can do right now is clean those up after a period of time.

If, on the other hand, your member IDs are meaningful, it's likely you will want to reuse these locations in the future when you need to get another lock on this member. The overhead for a node in ZK is very very low, so unless you are generating a ton of these nodes that you then never use again and never clean up, it's not likely to be too much of a problem.

We have discussed allowing "cleanup" type paths that will delete themselves when there are no child nodes. See:
https://issues.apache.org/jira/browse/ZOOKEEPER-723

C

-----Original Message-----
From: Jordan Zimmerman [mailto:jzimmerman@netflix.com] 
Sent: Tuesday, September 20, 2011 3:38 PM
To: user@zookeeper.apache.org
Subject: Lock recipes and the lock path

When writing the lock recipes for ZK, there is the potential for path
garbage to build up. For example, imagine you are writing a lock for some
kind go member ID. You'd have a base path for the locks (say
"/locks/members") and then append the member ID to get locks on it. Thus,
you'd end up with "/locks/members" building up paths, one for each member
ID. 

How can you safely clean up the path for each member ID lock? There
doesn't seem to be an atomic way to do it. If one process is cleaning up
for example "/locks/members/12345", it can't be sure than another process
might need to get a lock on that member ID. I.e. There's never a safe
point to delete "/locals/members/12345".

-JZ