You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2012/08/02 07:38:02 UTC
[jira] [Created] (HBASE-6496) Example ZK based scan policy
Lars Hofhansl created HBASE-6496:
------------------------------------
Summary: Example ZK based scan policy
Key: HBASE-6496
URL: https://issues.apache.org/jira/browse/HBASE-6496
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.96.0, 0.94.2
Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6496) Example ZK based scan policy
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430762#comment-13430762 ]
Zhihong Ted Yu edited comment on HBASE-6496 at 8/8/12 12:29 AM:
----------------------------------------------------------------
Unnecessary ZKWatcher creation / disposal can be avoided if you have a singleton ZKWatcher to be placed into shared data map first.
If the placement was successful, a new ZKWatcher can be created and replace the singleton for the same key.
was (Author: zhihyu@ebaysf.com):
ZKWatcher creation / disposal can be avoided if you have a singleton ZKWatcher to be placed into shared data map first.
If the placement was successful, a new ZKWatcher can be created and replace the singleton for the same key.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437056#comment-13437056 ]
stack commented on HBASE-6496:
------------------------------
+1 on commit. Add licenses on commit.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt, 6496-v2.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6496:
---------------------------------
Attachment: 6496.txt
Here's a work in progress.
It wasn't actually as straightforward as I thought, as there is no place within a RegionServer that would allow coprocessors to share some state.
Furthermore I could not use the RegionServer's ZooKeeperWatcher as there is no way to unregister a Listener, hence as RegionObservers come and go their listeners would pile up firing uselessly and preventing the RegionObserver implementation from the being GC'd.
So instead the RegionObserver itself is a watcher, which on other hand now leads to a Watcher per region.
I could use some advice here, will such use of ZK scale?
If it doesn't I could
# add a removeListenere method to ZooKeeperWatcher (would still need to work out how avoid many watches for the same node) or
# Have a way for RegionCoprocessorEnvironment to host some shared state for RegionObserver to coordinate in case such as this one. (could just be a Map that is accessible to all RegionObservers).
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427693#comment-13427693 ]
Andrew Purtell commented on HBASE-6496:
---------------------------------------
bq. CP classes are not unloaded, right?
Correct.
But they can be blacklisted via removal from the list of active loaded coprocessors if they throw an Error. See CoprocessorHost#handleCoprocessorThrowable.
The CoprocessorHost should clean up if the CP doesn't, or can't (due to deregistration).
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430762#comment-13430762 ]
Zhihong Ted Yu commented on HBASE-6496:
---------------------------------------
ZKWatcher creation / disposal can be avoided if you have a singleton ZKWatcher to be placed into shared data map first.
If the placement was successful, a new ZKWatcher can be created and replace the singleton for the same key.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431801#comment-13431801 ]
Zhihong Ted Yu commented on HBASE-6496:
---------------------------------------
@Lars:
The current way is fine too.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430877#comment-13430877 ]
Lars Hofhansl commented on HBASE-6496:
--------------------------------------
@Ted: I see...
There is no reliable way to have a singleton, though, since the instances could potentially have been loaded by separate classloaders.
Hence the containsKey() check to avoid creating a ZKWatcher unless necessary. The putIfAbsent just guards against the short race that another CP created the shared ZK between the contains check and the put.
Could also synchronize on this CP's sharedData; that would be the equivalent of a class lock. Taking the rare chance of unnecessarily creating a ZKWatcher seemed to be better approach.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427594#comment-13427594 ]
Andrew Purtell commented on HBASE-6496:
---------------------------------------
In an earlier iteration of CPs the coprocessor host provided an environment variable like facility, so for all CPs installed on a region could share state among themselves. This was a map provided by the CP host, so all CPs installed on a region could share state. We could bring that back as a map like you suggest but shared across all Observers in a RS, perhaps one map for every class that asks for it. (So, shared across the RS but private to each CP implementation.)
It could also be useful to have a controlled facility for ZK watchers anyway:
* Add to RegionServerServices a facility for getting ZK watchers on demand
* Add to the RegionCoprocessorEnvironment an API for getting ZK watchers from the RS, handing them back to be reaped, and dealing with freeing up dangling resources from any CP termination / unload.
* Extend above facility for shared watchers, one watcher per RS for a given CP, perhaps again keyed on class name.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427662#comment-13427662 ]
Andrew Purtell commented on HBASE-6496:
---------------------------------------
bq. How would that shared be exposed to a RegionObserver? Via the RegionCoprocessorEnvironment, or the CoprocessorContext?
I was thinking addition to interface CoprocessorEnvironment or RegionCoprocessorEnvironment and implementation in RegionCoprocessorHost.
Context is meant to hold state scoped to the invocation of a CP chain on a particular hook.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427687#comment-13427687 ]
Lars Hofhansl commented on HBASE-6496:
--------------------------------------
That makes sense. Then the only part is cleanup. CP classes are not unloaded, right?
So the onus would have to be on the CP implementation to the right thing in start/stop.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427689#comment-13427689 ]
Lars Hofhansl edited comment on HBASE-6496 at 8/2/12 10:39 PM:
---------------------------------------------------------------
I created HBASE-6505.
was (Author: lhofhansl):
I create HBASE-6505.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427634#comment-13427634 ]
Lars Hofhansl commented on HBASE-6496:
--------------------------------------
bq. Extend above facility for shared watchers, one watcher per RS for a given CP, perhaps again keyed on class name.
I like this one. In this case the hooks implemented by the region observer are in a critical path (at least preStoreScannerOpen is), so I do not want to call zk.getData(...) in that hook, and hence the need for a watcher to be asynchronously notified of changes.
Each RegionObserver instance would need to be notified about this, and if I understand the code right there is always exactly one instance per Region for which the observer is loaded. So sharing a single watcher with a listener for each CP would be ideal.
bq. We could bring that back as a map like you suggest but shared across all Observers in a RS, perhaps one map for every class that asks for it
That seems generally useful, and maybe even better suited for the problem mentioned above. A CP would check whether a watcher has been created (an object in that shared map), create one if not, and then add a listener. The watcher would need to support removing listeners.
With that CPs would be able to coordinate among themselves. It's not hard to make a ZK watcher, so the generality of this might be better.
We have found need for this in other projects, but so far have worked around it.
How would that shared be exposed to a RegionObserver? Via the RegionCoprocessorEnvironment, or the CoprocessorContext?
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435651#comment-13435651 ]
Lars Hofhansl commented on HBASE-6496:
--------------------------------------
OK... If there is no objections I would like to commit this soon.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt, 6496-v2.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430980#comment-13430980 ]
Zhihong Ted Yu commented on HBASE-6496:
---------------------------------------
bq. potentially have been loaded by separate classloaders
One singleton (marker) per classloader is not too bad, IMHO.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6496) Example ZK based scan policy
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430497#comment-13430497 ]
Zhihong Ted Yu edited comment on HBASE-6496 at 8/7/12 6:29 PM:
---------------------------------------------------------------
In process():
{code}
+ } catch (InterruptedException ix) {
+ } catch (KeeperException kx) {
{code}
Please restore interrupt status above.
License header missing in several files.
{code}
+ // there is a short race here
+ // in the worst case we create a watcher that will be notified once
+ re.getSharedData().putIfAbsent(
{code}
Should shared data provide putIfAbsent() functionality ?
was (Author: zhihyu@ebaysf.com):
What does the following call do in getData() ?
{code}
+ } else {
+ zk.exists(node, this);
+ }
{code}
License header missing in several files.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431606#comment-13431606 ]
Lars Hofhansl commented on HBASE-6496:
--------------------------------------
I don't think statics are advised in coprocessor, and thus providing an example using them is setting the wrong expectations.
Do you think the current way I have it is bad?
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430624#comment-13430624 ]
Zhihong Ted Yu commented on HBASE-6496:
---------------------------------------
w.r.t. putIfAbsent(), I meant if you use re.getSharedData().putIfAbsent() directly, you don't need the containsKey() check.
ZKWatcher needs to be disposed of in case an entry already existed in shared data.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430616#comment-13430616 ]
Lars Hofhansl commented on HBASE-6496:
--------------------------------------
Thanks Ted,
Yep, will add the license header. This was just to get a general feeling for whether the logic is even correct... Dealing with ZK correctly is tricky.
bq. Should shared data provide putIfAbsent() functionality ?
Not sure what you mean. It does (the interface it exposes is a ConcurrentMap as per HBASE-6505). Do you think it should not?
Will add Thread.interrupt (I keep forgetting about that :( ).
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430497#comment-13430497 ]
Zhihong Ted Yu commented on HBASE-6496:
---------------------------------------
What does the following call do in getData() ?
{code}
+ } else {
+ zk.exists(node, this);
+ }
{code}
License header missing in several files.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6496:
---------------------------------
Attachment: 6496-v2.txt
Here's a patch based on HBASE-6505.
All CP instances will use a single watcher, which keeps the date up to date asynchronously.
If the watcher get disconnected from ZK it will try reconnect periodically.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-v2.txt, 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427689#comment-13427689 ]
Lars Hofhansl commented on HBASE-6496:
--------------------------------------
I create HBASE-6505.
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6496) Example ZK based scan policy
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448296#comment-13448296 ]
Hudson commented on HBASE-6496:
-------------------------------
Integrated in HBase-0.94-security-on-Hadoop-23 #7 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/7/])
HBASE-6496 Addendum - add test category (Revision 1377177)
HBASE-6496 Example ZK based scan policy (Revision 1377155)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/example/TestZooKeeperScanPolicyObserver.java
larsh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/example
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ZooKeeperScanPolicyObserver.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/example
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/example/TestZooKeeperScanPolicyObserver.java
> Example ZK based scan policy
> ----------------------------
>
> Key: HBASE-6496
> URL: https://issues.apache.org/jira/browse/HBASE-6496
> Project: HBase
> Issue Type: Sub-task
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6496-0.94.txt, 6496-0.96.txt, 6496-0.96-v2.txt, 6496.txt, 6496-v2.txt
>
>
> Provide an example of a RegionServer that listens to a ZK node to learn about what set of KVs can safely be deleted during a compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira