You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2020/12/01 18:30:21 UTC
[GitHub] [accumulo] milleruntime opened a new pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
milleruntime opened a new pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819
* This patch replaces calls to get properties where a Zookeeper watch is
definitely not needed. The new method getWithoutWatch() will function
the same but not set a watch on the call to zookeeper exists(). This fix
will stop ZK watches from hanging around after a table is deleted.
* This patch will only work in 1.10 and cannot be merged forward to 2.x
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime closed pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
milleruntime closed pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime commented on pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
milleruntime commented on pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#issuecomment-736813053
I didn't think this fix touched parts of the code having to do with iterators... we tend to just get the per table classpath directly like here:
https://github.com/apache/accumulo/blob/3140f096f120883b71c75bf74e2b205ff086098a/server/base/src/main/java/org/apache/accumulo/server/conf/TableConfiguration.java#L216
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on a change in pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on a change in pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#discussion_r533714204
##########
File path: core/src/main/java/org/apache/accumulo/core/conf/Property.java
##########
@@ -1299,8 +1299,8 @@ public static boolean isClassProperty(String key) {
*/
public static <T> T createTableInstanceFromPropertyName(AccumuloConfiguration conf,
Property property, Class<T> base, T defaultInstance) {
- String clazzName = conf.get(property);
- String context = conf.get(TABLE_CLASSPATH);
+ String clazzName = conf.getWithoutWatch(property);
+ String context = conf.getWithoutWatch(TABLE_CLASSPATH);
Review comment:
Yes, but that was just an example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime commented on pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
milleruntime commented on pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#issuecomment-736813965
> I didn't think this fix touched parts of the code having to do with iterators... we tend to just get the per table classpath directly like here:
>
> https://github.com/apache/accumulo/blob/3140f096f120883b71c75bf74e2b205ff086098a/server/base/src/main/java/org/apache/accumulo/server/conf/TableConfiguration.java#L216
And here:
https://github.com/apache/accumulo/blob/3140f096f120883b71c75bf74e2b205ff086098a/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java#L516
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] jzgithub1 commented on pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
jzgithub1 commented on pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#issuecomment-736765218
Great idea @milleruntime! Glad that ZooCache isn't getting changed like it was in my PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on a change in pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on a change in pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#discussion_r533670396
##########
File path: server/base/src/main/java/org/apache/accumulo/server/conf/ZooCachePropertyAccessor.java
##########
@@ -103,6 +103,20 @@ String get(Property property, String path, AccumuloConfiguration parent) {
String key = property.getKey();
String value = get(path + "/" + key);
+ return checkValue(property, parent, key, value);
+ }
+
+ private String get(String path) {
+ byte[] v = propCache.get(path);
+ if (v != null) {
+ return new String(v, UTF_8);
+ } else {
+ return null;
+ }
Review comment:
```suggestion
return v == null ? null : new String(v, UTF_8);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime commented on pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
milleruntime commented on pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#issuecomment-736824338
I converted this to a draft since it is still risky for a bug fix version.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#issuecomment-736805760
> One big difference between this change and #1812 is that here we know exactly why we are making a call to ZooCache. For almost of all these occurrences, the table has to already exist. We won't ever call `new Tablet()` on a table that doesn't exist yet.
But these aren't just checking for properties existence of the table itself. We're also checking for properties which could be created later and whose non-existence could be cached if the watcher isn't set... like a per-table context classpath, which can change the behavior of how the next iterators are loaded for that table. If its non-existence is cached, and that cache is never updated, we will never see that this context classpath has been updated and could load the wrong iterator (or fail to load the iterator) the next time we try to read that property to load an iterator class.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime commented on pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
milleruntime commented on pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#issuecomment-736801932
One big difference between this change and #1812 is that here we know exactly why we are making a call to ZooCache. For almost of all these occurrences, the table has to already exist. We won't ever call `new Tablet()` on a table that doesn't exist yet.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] milleruntime commented on a change in pull request #1819: Make zk watch offenders call getWithoutWatch(). Fixes #1809
Posted by GitBox <gi...@apache.org>.
milleruntime commented on a change in pull request #1819:
URL: https://github.com/apache/accumulo/pull/1819#discussion_r533713216
##########
File path: core/src/main/java/org/apache/accumulo/core/conf/Property.java
##########
@@ -1299,8 +1299,8 @@ public static boolean isClassProperty(String key) {
*/
public static <T> T createTableInstanceFromPropertyName(AccumuloConfiguration conf,
Property property, Class<T> base, T defaultInstance) {
- String clazzName = conf.get(property);
- String context = conf.get(TABLE_CLASSPATH);
+ String clazzName = conf.getWithoutWatch(property);
+ String context = conf.getWithoutWatch(TABLE_CLASSPATH);
Review comment:
@ctubbsii is this what you are referring to for the per table context classpath? This method is called in these places:
- PerTableVolumeChooser.java for TABLE_VOLUME_CHOOSER
- Tablet.java for TABLE_COMPACTION_STRATEGY
- TabletServerResourceManager.java for TABLE_COMPACTION_STRATEGY
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org