You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "keith-turner (via GitHub)" <gi...@apache.org> on 2024/11/13 18:28:41 UTC
[PR] speeds up checking zoo locks for lots of server processes [accumulo]
keith-turner opened a new pull request, #5058:
URL: https://github.com/apache/accumulo/pull/5058
Many operations like listcompactions, listscans, admin serviceStatus check the zookeeper locks of server processes. These lock checks are currently done serially and each require a RPC to zookeeper. This change parallelizes the lock checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up checking zoo locks for lots of server processes [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner merged PR #5058:
URL: https://github.com/apache/accumulo/pull/5058
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up checking zoo locks for lots of server processes [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5058:
URL: https://github.com/apache/accumulo/pull/5058#discussion_r1841104010
##########
core/src/main/java/org/apache/accumulo/core/lock/ServiceLockPaths.java:
##########
@@ -455,14 +468,34 @@ private Set<ServiceLockPath> get(final String serverType,
// Dead TServers don't have lock data
results.add(slp);
} else {
- final ZcStat stat = new ZcStat();
- Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
- if (!sld.isEmpty()) {
- results.add(slp);
- }
+ // Execute reads to zookeeper to get lock info in parallel. The zookeeper client has
+ // a single shared connection to a server so this will not create lots of
+ // connections, it will place multiple outgoing request on that single zookeeper
+ // connection at the same time though.
+ futures.add(executor.submit(() -> {
+ final ZcStat stat = new ZcStat();
+ Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
+ if (sld.isPresent()) {
+ results.add(slp);
+ }
+ return null;
+ }));
}
}
}
+
+ // wait for futures to complete and check for errors
+ for (var future : futures) {
+ try {
+ future.get();
+ } catch (InterruptedException | ExecutionException e) {
+ throw new IllegalStateException(e);
+ }
+ }
+
+ if (executor != null) {
+ executor.shutdown();
+ }
Review Comment:
if an exception is thrown above, the executor does not get shut down?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up checking zoo locks for lots of server processes [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5058:
URL: https://github.com/apache/accumulo/pull/5058#discussion_r1841003260
##########
core/src/main/java/org/apache/accumulo/core/lock/ServiceLockPaths.java:
##########
@@ -455,14 +468,34 @@ private Set<ServiceLockPath> get(final String serverType,
// Dead TServers don't have lock data
results.add(slp);
} else {
- final ZcStat stat = new ZcStat();
- Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
- if (!sld.isEmpty()) {
- results.add(slp);
- }
+ // Execute reads to zookeeper to get lock info in parallel. The zookeeper client has
+ // a single shared connection to a server so this will not create lots of
+ // connections, it will place multiple outgoing request on that single zookeeper
+ // connection at the same time though.
+ futures.add(executor.submit(() -> {
+ final ZcStat stat = new ZcStat();
+ Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
+ if (sld.isPresent()) {
+ results.add(slp);
+ }
+ return null;
+ }));
}
}
}
+
+ // wait for futures to complete and check for errors
+ for (var future : futures) {
+ try {
+ future.get();
+ } catch (InterruptedException | ExecutionException e) {
+ throw new IllegalStateException(e);
+ }
+ }
Review Comment:
I don't think this will wait for all futures to return. Don't you need something like:
```
while (futures.size() > 0) {
Iterator<Future<?>> iter = futures.iterator();
while (iter.hasNext()) {
Future<?> future = iter.next();
if (future.isDone()) {
iter.remove();
try {
future.get();
} catch (InterruptedException | ExecutionException e) {
throw new IllegalStateException(e);
}
}
}
if (futures.size() > 0) {
UtilWaitThread.sleep(3_000);
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up checking zoo locks for lots of server processes [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on PR #5058:
URL: https://github.com/apache/accumulo/pull/5058#issuecomment-2474456131
Ran `mvn clean verify -Psunny` and it passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up checking zoo locks for lots of server processes [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5058:
URL: https://github.com/apache/accumulo/pull/5058#discussion_r1841319795
##########
core/src/main/java/org/apache/accumulo/core/lock/ServiceLockPaths.java:
##########
@@ -455,14 +468,34 @@ private Set<ServiceLockPath> get(final String serverType,
// Dead TServers don't have lock data
results.add(slp);
} else {
- final ZcStat stat = new ZcStat();
- Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
- if (!sld.isEmpty()) {
- results.add(slp);
- }
+ // Execute reads to zookeeper to get lock info in parallel. The zookeeper client has
+ // a single shared connection to a server so this will not create lots of
+ // connections, it will place multiple outgoing request on that single zookeeper
+ // connection at the same time though.
+ futures.add(executor.submit(() -> {
+ final ZcStat stat = new ZcStat();
+ Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
+ if (sld.isPresent()) {
+ results.add(slp);
+ }
+ return null;
+ }));
}
}
}
+
+ // wait for futures to complete and check for errors
+ for (var future : futures) {
+ try {
+ future.get();
+ } catch (InterruptedException | ExecutionException e) {
+ throw new IllegalStateException(e);
+ }
+ }
+
+ if (executor != null) {
+ executor.shutdown();
+ }
Review Comment:
yeah it does, I need to add a try/finally
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up checking zoo locks for lots of server processes [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5058:
URL: https://github.com/apache/accumulo/pull/5058#discussion_r1841319795
##########
core/src/main/java/org/apache/accumulo/core/lock/ServiceLockPaths.java:
##########
@@ -455,14 +468,34 @@ private Set<ServiceLockPath> get(final String serverType,
// Dead TServers don't have lock data
results.add(slp);
} else {
- final ZcStat stat = new ZcStat();
- Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
- if (!sld.isEmpty()) {
- results.add(slp);
- }
+ // Execute reads to zookeeper to get lock info in parallel. The zookeeper client has
+ // a single shared connection to a server so this will not create lots of
+ // connections, it will place multiple outgoing request on that single zookeeper
+ // connection at the same time though.
+ futures.add(executor.submit(() -> {
+ final ZcStat stat = new ZcStat();
+ Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
+ if (sld.isPresent()) {
+ results.add(slp);
+ }
+ return null;
+ }));
}
}
}
+
+ // wait for futures to complete and check for errors
+ for (var future : futures) {
+ try {
+ future.get();
+ } catch (InterruptedException | ExecutionException e) {
+ throw new IllegalStateException(e);
+ }
+ }
+
+ if (executor != null) {
+ executor.shutdown();
+ }
Review Comment:
yeah it does not, I need to add a try/finally
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up checking zoo locks for lots of server processes [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5058:
URL: https://github.com/apache/accumulo/pull/5058#discussion_r1842865247
##########
core/src/main/java/org/apache/accumulo/core/lock/ServiceLockPaths.java:
##########
@@ -455,14 +468,34 @@ private Set<ServiceLockPath> get(final String serverType,
// Dead TServers don't have lock data
results.add(slp);
} else {
- final ZcStat stat = new ZcStat();
- Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
- if (!sld.isEmpty()) {
- results.add(slp);
- }
+ // Execute reads to zookeeper to get lock info in parallel. The zookeeper client has
+ // a single shared connection to a server so this will not create lots of
+ // connections, it will place multiple outgoing request on that single zookeeper
+ // connection at the same time though.
+ futures.add(executor.submit(() -> {
+ final ZcStat stat = new ZcStat();
+ Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
+ if (sld.isPresent()) {
+ results.add(slp);
+ }
+ return null;
+ }));
}
}
}
+
+ // wait for futures to complete and check for errors
+ for (var future : futures) {
+ try {
+ future.get();
+ } catch (InterruptedException | ExecutionException e) {
+ throw new IllegalStateException(e);
+ }
+ }
+
+ if (executor != null) {
+ executor.shutdown();
+ }
Review Comment:
shutdown executor in try/finally in 0491bf9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up checking zoo locks for lots of server processes [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5058:
URL: https://github.com/apache/accumulo/pull/5058#discussion_r1841041830
##########
core/src/main/java/org/apache/accumulo/core/lock/ServiceLockPaths.java:
##########
@@ -455,14 +468,34 @@ private Set<ServiceLockPath> get(final String serverType,
// Dead TServers don't have lock data
results.add(slp);
} else {
- final ZcStat stat = new ZcStat();
- Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
- if (!sld.isEmpty()) {
- results.add(slp);
- }
+ // Execute reads to zookeeper to get lock info in parallel. The zookeeper client has
+ // a single shared connection to a server so this will not create lots of
+ // connections, it will place multiple outgoing request on that single zookeeper
+ // connection at the same time though.
+ futures.add(executor.submit(() -> {
+ final ZcStat stat = new ZcStat();
+ Optional<ServiceLockData> sld = ServiceLock.getLockData(cache, slp, stat);
+ if (sld.isPresent()) {
+ results.add(slp);
+ }
+ return null;
+ }));
}
}
}
+
+ // wait for futures to complete and check for errors
+ for (var future : futures) {
+ try {
+ future.get();
+ } catch (InterruptedException | ExecutionException e) {
+ throw new IllegalStateException(e);
+ }
+ }
Review Comment:
I think the Future.get() method blocks if the background task has not run or is running. The following is from its javadoc.
```
Waits if necessary for the computation to complete, and then retrieves its result.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org