You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by lzwei <ll...@gmail.com> on 2012/01/03 16:32:38 UTC
subscription problem with Zookeeper
I'm a freshman to ZK. We're going to build a multiple mounted file
system, all clients shares a centralized backend storage, with local
cache enabled. Since it is allow to read/write the backend file
concurrently, local data/meta data could be out-of-date, so the
subscription model is used to have notify data/meta-data changes,
other than check timestamp on each file access.
With ZK we could have threads watch znode for events and trigger
callbacks, but the lock granularity is a problem, the zk file
hierarchy represents exactly the same with our file system hierarchy,
but we can not watch all the znodes, is it possible for client
watching the one znode(for example the root znode), any changes on
child/low level znodes will notify clients, and envoke the callbacks.
It means only one watcher required for a given file system, which
reduces watchers dramatically.
Thanks a lot.
Re: zoo.cfg Server ID
Posted by Mahadev Konar <ma...@hortonworks.com>.
Hi Jordan,
From admin guide:
The myid file consists of a single line containing only the text of
that machine's id. So myid of server 1 would contain the text "1" and
nothing else. The id must be unique within the ensemble and should
have a value between 1 and 255
http://zookeeper.apache.org/doc/r3.4.2/zookeeperAdmin.html
hope that helps.
mahadev
On Wed, Jan 4, 2012 at 11:38 AM, Jordan Zimmerman
<jz...@netflix.com> wrote:
>
> If I read the code correctly, the server ID in zoo.cfg (i.e.
> "server.n=foo") does _not_ have to be sequential, 1 based, etc. i.e. it
> can be anything useful to me. For example, I could do:
>
> server.10064=foo:288:3888
> server.18=bar:288:3888
> server.65535=snafu:288:3888
>
>
> Am I correct?
>
> -JZ
>
zoo.cfg Server ID
Posted by Jordan Zimmerman <jz...@netflix.com>.
If I read the code correctly, the server ID in zoo.cfg (i.e.
"server.n=foo") does _not_ have to be sequential, 1 based, etc. i.e. it
can be anything useful to me. For example, I could do:
server.10064=foo:288:3888
server.18=bar:288:3888
server.65535=snafu:288:3888
Am I correct?
-JZ
Re: Multi doc?
Posted by Patrick Hunt <ph...@apache.org>.
If any of this needs more detail please do update the patch. Thanks.
On Wed, Jan 4, 2012 at 10:25 AM, Jordan Zimmerman
<jz...@netflix.com> wrote:
>>Did the patch for https://issues.apache.org/jira/browse/ZOOKEEPER-1336
>>not do a good enough job on that?
> Missed that - thanks.
>
>
>
> On 1/4/12 10:20 AM, "Patrick Hunt" <ph...@apache.org> wrote:
>
>>On Wed, Jan 4, 2012 at 10:03 AM, Jordan Zimmerman
>><jz...@netflix.com> wrote:
>>> * Can someone give more detail on Op.check()? I assume it generates an
>>> error if the version of the path doesn't match, but the doc is confusing
>>> to me. What happens if the transaction has both a setData and a check?
>>>
>>
>>I believe processing stops as soon as a check fails. We should clarify
>>that in the javadocs though.
>>
>>> * Does ZooKeeper.multi() guarantee that there will be exactly 1 OpResult
>>> for every Op in the transaction. Further, is there an ordering guarantee
>>> for the OpResult list - i.e. the OpResults are in the same order as the
>>> Ops argument? If this isn't so, I don't see how someone could correlate
>>>an
>>> Op to its result.
>>
>>Did the patch for https://issues.apache.org/jira/browse/ZOOKEEPER-1336
>>not do a good enough job on that?
>>
>>>
>>> On 1/4/12 7:30 AM, "Ted Dunning" <te...@gmail.com> wrote:
>>>
>>>>Not really.
>>>>
>>>>On Wed, Jan 4, 2012 at 12:47 AM, Jordan Zimmerman
>>>><jz...@netflix.com>wrote:
>>>>
>>>>> Is there documentation on the multi/transaction feature added in 3.4
>>>>>other
>>>>> than the Javadoc?
>>>>>
>>>>> -JZ
>>>>>
>>>>>
>>>
>>
>
Re: Multi doc?
Posted by Jordan Zimmerman <jz...@netflix.com>.
>Did the patch for https://issues.apache.org/jira/browse/ZOOKEEPER-1336
>not do a good enough job on that?
Missed that - thanks.
On 1/4/12 10:20 AM, "Patrick Hunt" <ph...@apache.org> wrote:
>On Wed, Jan 4, 2012 at 10:03 AM, Jordan Zimmerman
><jz...@netflix.com> wrote:
>> * Can someone give more detail on Op.check()? I assume it generates an
>> error if the version of the path doesn't match, but the doc is confusing
>> to me. What happens if the transaction has both a setData and a check?
>>
>
>I believe processing stops as soon as a check fails. We should clarify
>that in the javadocs though.
>
>> * Does ZooKeeper.multi() guarantee that there will be exactly 1 OpResult
>> for every Op in the transaction. Further, is there an ordering guarantee
>> for the OpResult list - i.e. the OpResults are in the same order as the
>> Ops argument? If this isn't so, I don't see how someone could correlate
>>an
>> Op to its result.
>
>Did the patch for https://issues.apache.org/jira/browse/ZOOKEEPER-1336
>not do a good enough job on that?
>
>>
>> On 1/4/12 7:30 AM, "Ted Dunning" <te...@gmail.com> wrote:
>>
>>>Not really.
>>>
>>>On Wed, Jan 4, 2012 at 12:47 AM, Jordan Zimmerman
>>><jz...@netflix.com>wrote:
>>>
>>>> Is there documentation on the multi/transaction feature added in 3.4
>>>>other
>>>> than the Javadoc?
>>>>
>>>> -JZ
>>>>
>>>>
>>
>
Re: Multi doc?
Posted by Patrick Hunt <ph...@apache.org>.
On Wed, Jan 4, 2012 at 10:03 AM, Jordan Zimmerman
<jz...@netflix.com> wrote:
> * Can someone give more detail on Op.check()? I assume it generates an
> error if the version of the path doesn't match, but the doc is confusing
> to me. What happens if the transaction has both a setData and a check?
>
I believe processing stops as soon as a check fails. We should clarify
that in the javadocs though.
> * Does ZooKeeper.multi() guarantee that there will be exactly 1 OpResult
> for every Op in the transaction. Further, is there an ordering guarantee
> for the OpResult list - i.e. the OpResults are in the same order as the
> Ops argument? If this isn't so, I don't see how someone could correlate an
> Op to its result.
Did the patch for https://issues.apache.org/jira/browse/ZOOKEEPER-1336
not do a good enough job on that?
>
> On 1/4/12 7:30 AM, "Ted Dunning" <te...@gmail.com> wrote:
>
>>Not really.
>>
>>On Wed, Jan 4, 2012 at 12:47 AM, Jordan Zimmerman
>><jz...@netflix.com>wrote:
>>
>>> Is there documentation on the multi/transaction feature added in 3.4
>>>other
>>> than the Javadoc?
>>>
>>> -JZ
>>>
>>>
>
Re: Multi doc?
Posted by Marshall McMullen <ma...@gmail.com>.
On Wed, Jan 4, 2012 at 3:21 PM, Ted Dunning <te...@gmail.com> wrote:
>
> > What happens if the transaction has both a setData and a check?
> >
>
> They happen in sequence, but I couldn't say exactly which version the check
> will see. I suspect that it will see the version before the setData is
> done.
>
>
It actually depends on the placement of the check in relation to the set
inside the multi-op. If the check is before the set, the check will see the
version before the set, but if the check is after the set, then it will see
the updated version after the set increments it.
Keep in mind two things here:
(1) You may not need the check call at all as the Set() call will fail if
the version number doesn't match the specified version in the Set() call.
(2) Operations inside a multi-op update the state of any zknodes in the
multi-op and all subsequent operations in the multi-op will see that new
state.
(3) If the multi-op fails, all changes are discarded and we roll back to
the prior state.
Re: Multi doc?
Posted by Ted Dunning <te...@gmail.com>.
On Wed, Jan 4, 2012 at 10:03 AM, Jordan Zimmerman <jz...@netflix.com>wrote:
> OK - a few questions:
>
> * Can someone give more detail on Op.check()? I assume it generates an
> error if the version of the path doesn't match, but the doc is confusing
> to me.
Correct.
> What happens if the transaction has both a setData and a check?
>
They happen in sequence, but I couldn't say exactly which version the check
will see. I suspect that it will see the version before the setData is
done.
> * Does ZooKeeper.multi() guarantee that there will be exactly 1 OpResult
> for every Op in the transaction. Further, is there an ordering guarantee
> for the OpResult list - i.e. the OpResults are in the same order as the
> Ops argument? If this isn't so, I don't see how someone could correlate an
> Op to its result.
>
Right. One to one correspondence is the contract.
Re: Multi doc?
Posted by Patrick Hunt <ph...@apache.org>.
On Wed, Jan 4, 2012 at 9:25 AM, Jordan Zimmerman <jz...@netflix.com> wrote:
>>Contributing docs is a great way to get started as a zk contributor
>>(hint hint ;-) )
>
> Aha! I already have. I updated the recipes doc to include GUIDs in locks.
Ok, you got me on that one. ;-) Thanks!
Patrick
Re: Multi doc?
Posted by Jordan Zimmerman <jz...@netflix.com>.
>Contributing docs is a great way to get started as a zk contributor
>(hint hint ;-) )
Aha! I already have. I updated the recipes doc to include GUIDs in locks.
On 1/4/12 9:23 AM, "Patrick Hunt" <ph...@apache.org> wrote:
>Also there's a pending patch to improve the multi javadoc:
>https://issues.apache.org/jira/browse/ZOOKEEPER-1336
>
>Contributing docs is a great way to get started as a zk contributor
>(hint hint ;-) )
>
>Patrick
>
>On Wed, Jan 4, 2012 at 7:30 AM, Ted Dunning <te...@gmail.com> wrote:
>> Not really.
>>
>> On Wed, Jan 4, 2012 at 12:47 AM, Jordan Zimmerman
>><jz...@netflix.com>wrote:
>>
>>> Is there documentation on the multi/transaction feature added in 3.4
>>>other
>>> than the Javadoc?
>>>
>>> -JZ
>>>
>>>
>
Re: Multi doc?
Posted by Patrick Hunt <ph...@apache.org>.
Also there's a pending patch to improve the multi javadoc:
https://issues.apache.org/jira/browse/ZOOKEEPER-1336
Contributing docs is a great way to get started as a zk contributor
(hint hint ;-) )
Patrick
On Wed, Jan 4, 2012 at 7:30 AM, Ted Dunning <te...@gmail.com> wrote:
> Not really.
>
> On Wed, Jan 4, 2012 at 12:47 AM, Jordan Zimmerman <jz...@netflix.com>wrote:
>
>> Is there documentation on the multi/transaction feature added in 3.4 other
>> than the Javadoc?
>>
>> -JZ
>>
>>
Re: Multi doc?
Posted by Jordan Zimmerman <jz...@netflix.com>.
OK - a few questions:
* Can someone give more detail on Op.check()? I assume it generates an
error if the version of the path doesn't match, but the doc is confusing
to me. What happens if the transaction has both a setData and a check?
* Does ZooKeeper.multi() guarantee that there will be exactly 1 OpResult
for every Op in the transaction. Further, is there an ordering guarantee
for the OpResult list - i.e. the OpResults are in the same order as the
Ops argument? If this isn't so, I don't see how someone could correlate an
Op to its result.
-JZ
On 1/4/12 7:30 AM, "Ted Dunning" <te...@gmail.com> wrote:
>Not really.
>
>On Wed, Jan 4, 2012 at 12:47 AM, Jordan Zimmerman
><jz...@netflix.com>wrote:
>
>> Is there documentation on the multi/transaction feature added in 3.4
>>other
>> than the Javadoc?
>>
>> -JZ
>>
>>
Re: Multi doc?
Posted by Ted Dunning <te...@gmail.com>.
Not really.
On Wed, Jan 4, 2012 at 12:47 AM, Jordan Zimmerman <jz...@netflix.com>wrote:
> Is there documentation on the multi/transaction feature added in 3.4 other
> than the Javadoc?
>
> -JZ
>
>
Multi doc?
Posted by Jordan Zimmerman <jz...@netflix.com>.
Is there documentation on the multi/transaction feature added in 3.4 other
than the Javadoc?
-JZ