You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@curator.apache.org by Jipeng Tan <ta...@gmail.com> on 2015/02/25 04:55:58 UTC
Is there a way to find which machine holds lease file created by InterProcessSemaphoreMutex
Hi,
We use InterProcessSemaphoreMutex to make sure certain machine only process
certain Task.
One issue I am currently facing is: sometimes a lease file for one Task
presents on Zookeeper, however, there is no machine process that Task after
zookeeper quorum is rebooted.
Since we have hundreds of machines, it is quite hard to go over every
machine's log to figure out which machines own the lease file.
So I wonder if there is a better way to find the association of machines
and lease files. Is that possible to add machines info (IP) to the Lock?
Thanks,
Jipeng
Re: Is there a way to find which machine holds lease file
created by InterProcessSemaphoreMutex
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Sorry, for some reason InterProcessSemaphoreMutex is not operating like the other Curator recipes. Please open a bug for this. It should be setting the data like other lock recipes.
-Jordan
On February 25, 2015 at 2:34:19 PM, Jipeng Tan (tan1986@gmail.com) wrote:
Hi Jordan,
The curator I am using is 2.7.0
Here is the snippet I used to acquire the lock:
lock = new InterProcessSemaphoreMutex(zkClient, task);
hasLock = lock.acquire(1, TimeUnit.SECONDS);
hasLock is true after I call lock.acquire.
What I observed after running above code there is no child under locks directory and only one file with no payload under leases directory.
I am not quite sure why no seems no locks. Can you plz give me some suggestions?
Thanks,
Jipeng
On Wed, Feb 25, 2015 at 11:20 AM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
Then you don’t have any locks it seems.
On February 25, 2015 at 2:19:41 PM, Jipeng Tan (tan1986@gmail.com) wrote:
Hi Jordan,
There is no child under locks directory:
[zk: localhost:2181(CONNECTED) 155] get /blah/TaskLocks/task-1/locks
cZxid = 0x1d301bc1ba
ctime = Thu Feb 19 17:18:30 GMT-07:00 2015
mZxid = 0x1d301bc1ba
mtime = Thu Feb 19 17:18:30 GMT-07:00 2015
pZxid = 0x1e00afb0ff
cversion = 342702
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
Lease file have no payload:
[zk: localhost:2181(CONNECTED) 6] get /indexer/33.8.717/active_80/locks/column-1/leases/_c_ff76b8f0-2fb8-44a8-9ebc-e3462f89ef94-lease-0001746006
cZxid = 0x1e0000ff93
ctime = Tue Feb 24 17:58:45 GMT-07:00 2015
mZxid = 0x1e0000ff93
mtime = Tue Feb 24 17:58:45 GMT-07:00 2015
pZxid = 0x1e0000ff93
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x54a5fbd58ea0133
dataLength = 0
numChildren = 0
I also attached the code to create lock and acquire lock:
private InterProcessSemaphoreMutex lock = null;
....
private boolean acquireLock(final String taskName) throws TaskAcquisitionException {
....
lock = new InterProcessSemaphoreMutex(zkClient, task);
boolean hasLock = false;
try {
hasLock = lock.acquire(1, TimeUnit.SECONDS);
} catch (Exception e) {
throw new TaskAcquisitionException("Unable to acquire a lock for " + task, e);
}
return hasLock;
}
@Override
public synchronized void stateChanged(final CuratorFramework client, final ConnectionState newState) {
switch (newState) {
case LOST:
case SUSPENDED:
try {
if (lock != null) {
if (lock.isAcquiredInThisProcess()) {
LOGGER.info("Releasing lock.");
lock.release();
LOGGER.info("Lock released.");
} else {
LOGGER.info("No Lock is owned. Nothing to be released.");
}
}
} catch (Exception e) {
LOGGER.error("Error releasing lock.", e);
}
break;
case RECONNECTED:
break;
default:
break;
}
}
Thanks,
Jipeng
On Wed, Feb 25, 2015 at 7:06 AM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
You’re getting the data from the parent. It’s the lock/lease nodes that have the data.
-Jordan
On February 25, 2015 at 4:07:48 AM, Jipeng Tan (tan1986@gmail.com) wrote:
Thanks for the promptly response.
Can I do this in zookeeper CLI?
Here is my zookeeper directory structure I have:
/blah/TaskLocks/task-1/
/blah/TaskLocks/task-2/
/blah/TaskLocks/task-3/
....
/blah/TaskLocks/task-n/
task-1...n is defined as the root of my task directory.
I believe curator creates two more directories (locks and leases) under task-1...n
I first try yo get payload from the child from locks directory. However, there is no data.
[zk: localhost:2181(CONNECTED) 155] get /blah/TaskLocks/task-1/locks
cZxid = 0x1d301bc1ba
ctime = Thu Feb 19 17:18:30 GMT-07:00 2015
mZxid = 0x1d301bc1ba
mtime = Thu Feb 19 17:18:30 GMT-07:00 2015
pZxid = 0x1e00afb0ff
cversion = 342702
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
Did I do something wrong?
Thanks,
Jipeng
On Tue, Feb 24, 2015 at 8:05 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
Unless you change it in the factory, the IP of the instance _is_ stored as the payload for the lock file. You can getChildren on the lock parent, sort them and get the payload of the first sorted child.
-Jordan
On February 24, 2015 at 10:58:16 PM, Jipeng Tan (tan1986@gmail.com) wrote:
Hi,
We use InterProcessSemaphoreMutex to make sure certain machine only process certain Task.
One issue I am currently facing is: sometimes a lease file for one Task presents on Zookeeper, however, there is no machine process that Task after zookeeper quorum is rebooted.
Since we have hundreds of machines, it is quite hard to go over every machine's log to figure out which machines own the lease file.
So I wonder if there is a better way to find the association of machines and lease files. Is that possible to add machines info (IP) to the Lock?
Thanks,
Jipeng
Re: Is there a way to find which machine holds lease file
created by InterProcessSemaphoreMutex
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Then you don’t have any locks it seems.
On February 25, 2015 at 2:19:41 PM, Jipeng Tan (tan1986@gmail.com) wrote:
Hi Jordan,
There is no child under locks directory:
[zk: localhost:2181(CONNECTED) 155] get /blah/TaskLocks/task-1/locks
cZxid = 0x1d301bc1ba
ctime = Thu Feb 19 17:18:30 GMT-07:00 2015
mZxid = 0x1d301bc1ba
mtime = Thu Feb 19 17:18:30 GMT-07:00 2015
pZxid = 0x1e00afb0ff
cversion = 342702
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
Lease file have no payload:
[zk: localhost:2181(CONNECTED) 6] get /indexer/33.8.717/active_80/locks/column-1/leases/_c_ff76b8f0-2fb8-44a8-9ebc-e3462f89ef94-lease-0001746006
cZxid = 0x1e0000ff93
ctime = Tue Feb 24 17:58:45 GMT-07:00 2015
mZxid = 0x1e0000ff93
mtime = Tue Feb 24 17:58:45 GMT-07:00 2015
pZxid = 0x1e0000ff93
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x54a5fbd58ea0133
dataLength = 0
numChildren = 0
I also attached the code to create lock and acquire lock:
private InterProcessSemaphoreMutex lock = null;
....
private boolean acquireLock(final String taskName) throws TaskAcquisitionException {
....
lock = new InterProcessSemaphoreMutex(zkClient, task);
boolean hasLock = false;
try {
hasLock = lock.acquire(1, TimeUnit.SECONDS);
} catch (Exception e) {
throw new TaskAcquisitionException("Unable to acquire a lock for " + task, e);
}
return hasLock;
}
@Override
public synchronized void stateChanged(final CuratorFramework client, final ConnectionState newState) {
switch (newState) {
case LOST:
case SUSPENDED:
try {
if (lock != null) {
if (lock.isAcquiredInThisProcess()) {
LOGGER.info("Releasing lock.");
lock.release();
LOGGER.info("Lock released.");
} else {
LOGGER.info("No Lock is owned. Nothing to be released.");
}
}
} catch (Exception e) {
LOGGER.error("Error releasing lock.", e);
}
break;
case RECONNECTED:
break;
default:
break;
}
}
Thanks,
Jipeng
On Wed, Feb 25, 2015 at 7:06 AM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
You’re getting the data from the parent. It’s the lock/lease nodes that have the data.
-Jordan
On February 25, 2015 at 4:07:48 AM, Jipeng Tan (tan1986@gmail.com) wrote:
Thanks for the promptly response.
Can I do this in zookeeper CLI?
Here is my zookeeper directory structure I have:
/blah/TaskLocks/task-1/
/blah/TaskLocks/task-2/
/blah/TaskLocks/task-3/
....
/blah/TaskLocks/task-n/
task-1...n is defined as the root of my task directory.
I believe curator creates two more directories (locks and leases) under task-1...n
I first try yo get payload from the child from locks directory. However, there is no data.
[zk: localhost:2181(CONNECTED) 155] get /blah/TaskLocks/task-1/locks
cZxid = 0x1d301bc1ba
ctime = Thu Feb 19 17:18:30 GMT-07:00 2015
mZxid = 0x1d301bc1ba
mtime = Thu Feb 19 17:18:30 GMT-07:00 2015
pZxid = 0x1e00afb0ff
cversion = 342702
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
Did I do something wrong?
Thanks,
Jipeng
On Tue, Feb 24, 2015 at 8:05 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
Unless you change it in the factory, the IP of the instance _is_ stored as the payload for the lock file. You can getChildren on the lock parent, sort them and get the payload of the first sorted child.
-Jordan
On February 24, 2015 at 10:58:16 PM, Jipeng Tan (tan1986@gmail.com) wrote:
Hi,
We use InterProcessSemaphoreMutex to make sure certain machine only process certain Task.
One issue I am currently facing is: sometimes a lease file for one Task presents on Zookeeper, however, there is no machine process that Task after zookeeper quorum is rebooted.
Since we have hundreds of machines, it is quite hard to go over every machine's log to figure out which machines own the lease file.
So I wonder if there is a better way to find the association of machines and lease files. Is that possible to add machines info (IP) to the Lock?
Thanks,
Jipeng
Re: Is there a way to find which machine holds lease file created by InterProcessSemaphoreMutex
Posted by Jipeng Tan <ta...@gmail.com>.
Hi Jordan,
There is no child under locks directory:
[zk: localhost:2181(CONNECTED) 155] get /blah/TaskLocks/task-1/locks
cZxid = 0x1d301bc1ba
ctime = Thu Feb 19 17:18:30 GMT-07:00 2015
mZxid = 0x1d301bc1ba
mtime = Thu Feb 19 17:18:30 GMT-07:00 2015
pZxid = 0x1e00afb0ff
cversion = 342702
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
*numChildren = 0*
Lease file have no payload:
[zk: localhost:2181(CONNECTED) 6] get
/indexer/33.8.717/active_80/locks/column-1/leases/_c_ff76b8f0-2fb8-44a8-9ebc-e3462f89ef94-lease-0001746006
cZxid = 0x1e0000ff93
ctime = Tue Feb 24 17:58:45 GMT-07:00 2015
mZxid = 0x1e0000ff93
mtime = Tue Feb 24 17:58:45 GMT-07:00 2015
pZxid = 0x1e0000ff93
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x54a5fbd58ea0133
dataLength = 0
numChildren = 0
I also attached the code to create lock and acquire lock:
private InterProcessSemaphoreMutex lock = null;
....
private boolean acquireLock(final String taskName) throws
TaskAcquisitionException {
....
lock = new InterProcessSemaphoreMutex(zkClient, task);
boolean hasLock = false;
try {
hasLock = lock.acquire(1, TimeUnit.SECONDS);
} catch (Exception e) {
throw new TaskAcquisitionException("Unable to acquire a lock
for " + task, e);
}
return hasLock;
}
@Override
public synchronized void stateChanged(final CuratorFramework client,
final ConnectionState newState) {
switch (newState) {
case LOST:
case SUSPENDED:
try {
if (lock != null) {
if (lock.isAcquiredInThisProcess()) {
LOGGER.info("Releasing lock.");
lock.release();
LOGGER.info("Lock released.");
} else {
LOGGER.info("No Lock is owned. Nothing to be
released.");
}
}
} catch (Exception e) {
LOGGER.error("Error releasing lock.", e);
}
break;
case RECONNECTED:
break;
default:
break;
}
}
Thanks,
Jipeng
On Wed, Feb 25, 2015 at 7:06 AM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> You’re getting the data from the parent. It’s the lock/lease nodes that
> have the data.
>
> -Jordan
>
>
>
> On February 25, 2015 at 4:07:48 AM, Jipeng Tan (tan1986@gmail.com) wrote:
>
> Thanks for the promptly response.
>
> Can I do this in zookeeper CLI?
>
> Here is my zookeeper directory structure I have:
>
> /blah/TaskLocks/task-1/
> /blah/TaskLocks/task-2/
> /blah/TaskLocks/task-3/
> ....
> /blah/TaskLocks/task-n/
>
> task-1...n is defined as the root of my task directory.
>
> I believe curator creates two more directories (locks and leases) under
> task-1...n
>
> I first try yo get payload from the child from locks directory. However,
> there is no data.
>
>
> [zk: localhost:2181(CONNECTED) 155] get /blah/TaskLocks/task-1/locks
>
> cZxid = 0x1d301bc1ba
> ctime = Thu Feb 19 17:18:30 GMT-07:00 2015
> mZxid = 0x1d301bc1ba
> mtime = Thu Feb 19 17:18:30 GMT-07:00 2015
> pZxid = 0x1e00afb0ff
> cversion = 342702
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 0
> *numChildren = 0*
>
> Did I do something wrong?
>
>
> Thanks,
> Jipeng
>
> On Tue, Feb 24, 2015 at 8:05 PM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> Unless you change it in the factory, the IP of the instance _is_ stored
>> as the payload for the lock file. You can getChildren on the lock parent,
>> sort them and get the payload of the first sorted child.
>>
>> -Jordan
>>
>>
>>
>> On February 24, 2015 at 10:58:16 PM, Jipeng Tan (tan1986@gmail.com)
>> wrote:
>>
>> Hi,
>>
>> We use InterProcessSemaphoreMutex to make sure certain machine only
>> process certain Task.
>>
>> One issue I am currently facing is: sometimes a lease file for one Task
>> presents on Zookeeper, however, there is no machine process that Task after
>> zookeeper quorum is rebooted.
>>
>> Since we have hundreds of machines, it is quite hard to go over every
>> machine's log to figure out which machines own the lease file.
>>
>> So I wonder if there is a better way to find the association of machines
>> and lease files. Is that possible to add machines info (IP) to the Lock?
>>
>> Thanks,
>> Jipeng
>>
>>
>
Re: Is there a way to find which machine holds lease file
created by InterProcessSemaphoreMutex
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
You’re getting the data from the parent. It’s the lock/lease nodes that have the data.
-Jordan
On February 25, 2015 at 4:07:48 AM, Jipeng Tan (tan1986@gmail.com) wrote:
Thanks for the promptly response.
Can I do this in zookeeper CLI?
Here is my zookeeper directory structure I have:
/blah/TaskLocks/task-1/
/blah/TaskLocks/task-2/
/blah/TaskLocks/task-3/
....
/blah/TaskLocks/task-n/
task-1...n is defined as the root of my task directory.
I believe curator creates two more directories (locks and leases) under task-1...n
I first try yo get payload from the child from locks directory. However, there is no data.
[zk: localhost:2181(CONNECTED) 155] get /blah/TaskLocks/task-1/locks
cZxid = 0x1d301bc1ba
ctime = Thu Feb 19 17:18:30 GMT-07:00 2015
mZxid = 0x1d301bc1ba
mtime = Thu Feb 19 17:18:30 GMT-07:00 2015
pZxid = 0x1e00afb0ff
cversion = 342702
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
Did I do something wrong?
Thanks,
Jipeng
On Tue, Feb 24, 2015 at 8:05 PM, Jordan Zimmerman <jo...@jordanzimmerman.com> wrote:
Unless you change it in the factory, the IP of the instance _is_ stored as the payload for the lock file. You can getChildren on the lock parent, sort them and get the payload of the first sorted child.
-Jordan
On February 24, 2015 at 10:58:16 PM, Jipeng Tan (tan1986@gmail.com) wrote:
Hi,
We use InterProcessSemaphoreMutex to make sure certain machine only process certain Task.
One issue I am currently facing is: sometimes a lease file for one Task presents on Zookeeper, however, there is no machine process that Task after zookeeper quorum is rebooted.
Since we have hundreds of machines, it is quite hard to go over every machine's log to figure out which machines own the lease file.
So I wonder if there is a better way to find the association of machines and lease files. Is that possible to add machines info (IP) to the Lock?
Thanks,
Jipeng
Re: Is there a way to find which machine holds lease file created by InterProcessSemaphoreMutex
Posted by Jipeng Tan <ta...@gmail.com>.
Thanks for the promptly response.
Can I do this in zookeeper CLI?
Here is my zookeeper directory structure I have:
/blah/TaskLocks/task-1/
/blah/TaskLocks/task-2/
/blah/TaskLocks/task-3/
....
/blah/TaskLocks/task-n/
task-1...n is defined as the root of my task directory.
I believe curator creates two more directories (locks and leases) under
task-1...n
I first try yo get payload from the child from locks directory. However,
there is no data.
[zk: localhost:2181(CONNECTED) 155] get /blah/TaskLocks/task-1/locks
cZxid = 0x1d301bc1ba
ctime = Thu Feb 19 17:18:30 GMT-07:00 2015
mZxid = 0x1d301bc1ba
mtime = Thu Feb 19 17:18:30 GMT-07:00 2015
pZxid = 0x1e00afb0ff
cversion = 342702
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
*numChildren = 0*
Did I do something wrong?
Thanks,
Jipeng
On Tue, Feb 24, 2015 at 8:05 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:
> Unless you change it in the factory, the IP of the instance _is_ stored as
> the payload for the lock file. You can getChildren on the lock parent, sort
> them and get the payload of the first sorted child.
>
> -Jordan
>
>
>
> On February 24, 2015 at 10:58:16 PM, Jipeng Tan (tan1986@gmail.com) wrote:
>
> Hi,
>
> We use InterProcessSemaphoreMutex to make sure certain machine only
> process certain Task.
>
> One issue I am currently facing is: sometimes a lease file for one Task
> presents on Zookeeper, however, there is no machine process that Task after
> zookeeper quorum is rebooted.
>
> Since we have hundreds of machines, it is quite hard to go over every
> machine's log to figure out which machines own the lease file.
>
> So I wonder if there is a better way to find the association of machines
> and lease files. Is that possible to add machines info (IP) to the Lock?
>
> Thanks,
> Jipeng
>
>
Re: Is there a way to find which machine holds lease file
created by InterProcessSemaphoreMutex
Posted by Jordan Zimmerman <jo...@jordanzimmerman.com>.
Unless you change it in the factory, the IP of the instance _is_ stored as the payload for the lock file. You can getChildren on the lock parent, sort them and get the payload of the first sorted child.
-Jordan
On February 24, 2015 at 10:58:16 PM, Jipeng Tan (tan1986@gmail.com) wrote:
Hi,
We use InterProcessSemaphoreMutex to make sure certain machine only process certain Task.
One issue I am currently facing is: sometimes a lease file for one Task presents on Zookeeper, however, there is no machine process that Task after zookeeper quorum is rebooted.
Since we have hundreds of machines, it is quite hard to go over every machine's log to figure out which machines own the lease file.
So I wonder if there is a better way to find the association of machines and lease files. Is that possible to add machines info (IP) to the Lock?
Thanks,
Jipeng