You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "keith-turner (via GitHub)" <gi...@apache.org> on 2025/01/15 00:38:35 UTC
[PR] speeds up fate lock acquisition [accumulo]
keith-turner opened a new pull request, #5262:
URL: https://github.com/apache/accumulo/pull/5262
Stores the lock data for fate locks in the zookeeper node name instead of the zookeeper data for the node. Ran some local performance test with hundreds of fate operations and saw lock times go from 750ms to 15ms.
fixes #5181
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1938321152
##########
core/src/main/java/org/apache/accumulo/core/fate/AdminUtil.java:
##########
@@ -275,38 +277,36 @@ private void findLocks(ZooSession zk, final ServiceLockPath lockPath,
List<String> lockedIds = zr.getChildren(lockPath.toString());
for (String id : lockedIds) {
-
try {
-
FateLockPath fLockPath = FateLock.path(lockPath + "/" + id);
- List<String> lockNodes =
- FateLock.validateAndSort(fLockPath, zr.getChildren(fLockPath.toString()));
+ SortedSet<FateLock.NodeName> lockNodes =
+ FateLock.validateAndWarn(fLockPath, zr.getChildren(fLockPath.toString()));
int pos = 0;
boolean sawWriteLock = false;
- for (String node : lockNodes) {
+ for (FateLock.NodeName node : lockNodes) {
try {
- byte[] data = zr.getData(lockPath + "/" + id + "/" + node);
- // Example data: "READ:<FateId>". FateId contains ':' hence the limit of 2
- String[] lda = new String(data, UTF_8).split(":", 2);
- FateId fateId = FateId.from(lda[1]);
+ FateLock.FateLockEntry fateLockEntry = node.fateLockEntry.get();
+ var fateId = fateLockEntry.getFateId();
+ var lockType = fateLockEntry.getLockType();
- if (lda[0].charAt(0) == 'W') {
+ if (lockType == LockType.WRITE) {
sawWriteLock = true;
}
Map<FateId,List<String>> locks;
if (pos == 0) {
locks = heldLocks;
- } else if (lda[0].charAt(0) == 'R' && !sawWriteLock) {
+ } else if (lockType == LockType.READ && !sawWriteLock) {
locks = heldLocks;
} else {
locks = waitingLocks;
}
- locks.computeIfAbsent(fateId, k -> new ArrayList<>()).add(lda[0].charAt(0) + ":" + id);
+ locks.computeIfAbsent(fateId, k -> new ArrayList<>())
Review Comment:
I suspect the new FateLockEntry type could be used in this Admin class instead of the string things its doing here. Going to open a follow on issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1928797707
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -78,11 +78,11 @@ public static class FateLockNode implements Comparable<FateLockNode> {
public final long sequence;
public final String lockData;
- private FateLockNode(String nodeName) {
+ FateLockNode(String nodeName) {
int len = nodeName.length();
Preconditions.checkArgument(nodeName.startsWith(PREFIX) && nodeName.charAt(len - 11) == '#',
"Illegal node name %s", nodeName);
Review Comment:
In the case of an overflow, the character at (len - 11) be a minus sign and cause this to fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1917149068
##########
test/src/main/java/org/apache/accumulo/test/compaction/ExternalCompaction_1_IT.java:
##########
@@ -500,6 +504,68 @@ public void testManytablets() throws Exception {
compact(client, table1, 3, GROUP4, true);
verify(client, table1, 3);
+
+ List<TabletId> tabletIds;
+ // start a compaction on each tablet
+ try (var tablets = client.tableOperations().getTabletInformation(table1, new Range())) {
Review Comment:
I wrote this test initially to check performance, but realized its a good test of many compaction on the same table w/ different config also.
I did not initially see a performance difference w/ this test. Looking into it the reason was the compaction were finishing too quickly, so only 8 or 9 compaction fate operations were active at a time. For the performance bug to be observed needed many more active fate operations. Locally I modified the test to kill the compactor processes and start 1K compaction fate operations. With those changes I could see orders of magnitude diffs for fate lock acquisition time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner merged PR #5262:
URL: https://github.com/apache/accumulo/pull/5262
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1920493805
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -68,16 +74,56 @@ public FateLock(ZooReaderWriter zrw, FateLockPath path) {
this.path = requireNonNull(path);
}
+ public static class FateLockNode implements Comparable<FateLockNode> {
+ public final long sequence;
+ public final String lockData;
+
+ private FateLockNode(String nodeName) {
+ int len = nodeName.length();
+ Preconditions.checkArgument(nodeName.startsWith(PREFIX) && nodeName.charAt(len - 11) == '#',
+ "Illegal node name %s", nodeName);
+ sequence = Long.parseLong(nodeName.substring(len - 10));
+ lockData = nodeName.substring(PREFIX.length(), len - 11);
Review Comment:
So, while this is probably unlikely to happen, the sequential numbering can overflow causing a negative to occur, which I think would cause a problem with this parsing instead of splitting on `#`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1938333511
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -76,26 +77,10 @@ private FateLockEntry(LockType lockType, FateId fateId) {
this.fateId = Objects.requireNonNull(fateId);
}
- private FateLockEntry(byte[] entry) {
- if (entry == null || entry.length < 1) {
- throw new IllegalArgumentException();
- }
-
- int split = -1;
- for (int i = 0; i < entry.length; i++) {
- if (entry[i] == ':') {
- split = i;
- break;
- }
- }
-
- if (split == -1) {
- throw new IllegalArgumentException();
- }
-
- this.lockType = LockType.valueOf(new String(entry, 0, split, UTF_8));
- this.fateId =
- FateId.from(new String(Arrays.copyOfRange(entry, split + 1, entry.length), UTF_8));
+ private FateLockEntry(String entry) {
+ var fields = entry.split(":", 2);
Review Comment:
That just grabs the first two splits even if there are more. The FateId also has a `:` so calling split w/o the 2 will return three fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1920805513
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -68,16 +74,56 @@ public FateLock(ZooReaderWriter zrw, FateLockPath path) {
this.path = requireNonNull(path);
}
+ public static class FateLockNode implements Comparable<FateLockNode> {
+ public final long sequence;
+ public final String lockData;
+
+ private FateLockNode(String nodeName) {
+ int len = nodeName.length();
+ Preconditions.checkArgument(nodeName.startsWith(PREFIX) && nodeName.charAt(len - 11) == '#',
+ "Illegal node name %s", nodeName);
+ sequence = Long.parseLong(nodeName.substring(len - 10));
+ lockData = nodeName.substring(PREFIX.length(), len - 11);
Review Comment:
Added handing and test for negative seq numbers in 2309b9b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1915815637
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -68,16 +72,34 @@ public FateLock(ZooReaderWriter zrw, FateLockPath path) {
this.path = requireNonNull(path);
}
+ public static class FateLockNode {
+ public final long sequence;
+ public final String lockData;
+
+ private FateLockNode(String nodeName) {
+ int len = nodeName.length();
+ Preconditions.checkArgument(nodeName.startsWith(PREFIX) && nodeName.charAt(len - 11) == '#',
+ "Illegal node name %s", nodeName);
+ sequence = Long.parseLong(nodeName.substring(len - 10));
+ lockData = nodeName.substring(PREFIX.length(), len - 11);
+ }
+ }
+
+ // TODO change data arg from byte[] to String.. in the rest of the code its always a String.
@Override
public long addEntry(byte[] data) {
+
+ String dataString = new String(data, UTF_8);
+ Preconditions.checkState(!dataString.contains("#"));
+
String newPath;
try {
while (true) {
try {
- newPath = zoo.putPersistentSequential(path + "/" + PREFIX, data);
+ newPath = zoo.putPersistentSequential(path + "/" + PREFIX + dataString + "#", data);
Review Comment:
```suggestion
newPath = zoo.putPersistentSequential(path + "/" + PREFIX + dataString + "#", new byte[0]);
```
##########
core/src/main/java/org/apache/accumulo/core/fate/AdminUtil.java:
##########
@@ -285,19 +286,19 @@ private void findLocks(ZooSession zk, final ServiceLockPath lockPath,
List<String> lockedIds = zr.getChildren(lockPath.toString());
for (String id : lockedIds) {
-
try {
-
FateLockPath fLockPath = FateLock.path(lockPath + "/" + id);
- List<String> lockNodes =
- FateLock.validateAndSort(fLockPath, zr.getChildren(fLockPath.toString()));
+ List<FateLock.FateLockNode> lockNodes =
+ FateLock.validateAndWarn(fLockPath, zr.getChildren(fLockPath.toString()));
+
+ lockNodes.sort(Comparator.comparingLong(ln -> ln.sequence));
Review Comment:
Should the return type be a sorted set instead of a list that you sort later?
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/DistributedReadWriteLock.java:
##########
@@ -251,19 +253,28 @@ public DistributedReadWriteLock(QueueLock qlock, byte[] data) {
}
public static DistributedLock recoverLock(QueueLock qlock, byte[] data) {
- SortedMap<Long,byte[]> entries = qlock.getEarlierEntries(Long.MAX_VALUE);
- for (Entry<Long,byte[]> entry : entries.entrySet()) {
- ParsedLock parsed = new ParsedLock(entry.getValue());
- if (Arrays.equals(data, parsed.getUserData())) {
+ SortedMap<Long,byte[]> entries = qlock.getEntries((seq, lockData) -> {
+ ParsedLock parsed = new ParsedLock(lockData);
+ return Arrays.equals(data, parsed.getUserData());
+ });
+
+ switch (entries.size()) {
+ case 0:
+ return null;
+ case 1:
+ var entry = entries.entrySet().iterator().next();
+ ParsedLock parsed = new ParsedLock(entry.getValue());
switch (parsed.getType()) {
case READ:
return new ReadLock(qlock, parsed.getUserData(), entry.getKey());
case WRITE:
return new WriteLock(qlock, parsed.getUserData(), entry.getKey());
+ default:
+ throw new IllegalStateException("Uknown lock type " + parsed.getType());
Review Comment:
```suggestion
throw new IllegalStateException("Unknown lock type " + parsed.getType());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1929004078
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -78,11 +78,11 @@ public static class FateLockNode implements Comparable<FateLockNode> {
public final long sequence;
public final String lockData;
- private FateLockNode(String nodeName) {
+ FateLockNode(String nodeName) {
int len = nodeName.length();
Preconditions.checkArgument(nodeName.startsWith(PREFIX) && nodeName.charAt(len - 11) == '#',
"Illegal node name %s", nodeName);
Review Comment:
For the case Long.parseUnsignedLong catches the problem it throws a more specific exception, I updated the test to check for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1928991608
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -78,11 +78,11 @@ public static class FateLockNode implements Comparable<FateLockNode> {
public final long sequence;
public final String lockData;
- private FateLockNode(String nodeName) {
+ FateLockNode(String nodeName) {
int len = nodeName.length();
Preconditions.checkArgument(nodeName.startsWith(PREFIX) && nodeName.charAt(len - 11) == '#',
"Illegal node name %s", nodeName);
Review Comment:
It depends on the number of digits in the negative number. If its a 10 digit negative number then `%010d` will produce a 11 char string. If its 9 digits or less then `%010d` will produce a 10 char string including the negative. For the 9 digit or less case the `#` check will not catch negative, but the later `Long.parseUnsignedLong` call will catch it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1917140889
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -68,16 +72,34 @@ public FateLock(ZooReaderWriter zrw, FateLockPath path) {
this.path = requireNonNull(path);
}
+ public static class FateLockNode {
+ public final long sequence;
+ public final String lockData;
+
+ private FateLockNode(String nodeName) {
+ int len = nodeName.length();
+ Preconditions.checkArgument(nodeName.startsWith(PREFIX) && nodeName.charAt(len - 11) == '#',
+ "Illegal node name %s", nodeName);
+ sequence = Long.parseLong(nodeName.substring(len - 10));
+ lockData = nodeName.substring(PREFIX.length(), len - 11);
+ }
+ }
+
+ // TODO change data arg from byte[] to String.. in the rest of the code its always a String.
Review Comment:
Removed the TODO and #5264. In the TODO I was thinking of using String, but realized using more concrete types would be better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1915798021
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -68,16 +72,34 @@ public FateLock(ZooReaderWriter zrw, FateLockPath path) {
this.path = requireNonNull(path);
}
+ public static class FateLockNode {
+ public final long sequence;
+ public final String lockData;
+
+ private FateLockNode(String nodeName) {
+ int len = nodeName.length();
+ Preconditions.checkArgument(nodeName.startsWith(PREFIX) && nodeName.charAt(len - 11) == '#',
+ "Illegal node name %s", nodeName);
+ sequence = Long.parseLong(nodeName.substring(len - 10));
+ lockData = nodeName.substring(PREFIX.length(), len - 11);
+ }
+ }
+
+ // TODO change data arg from byte[] to String.. in the rest of the code its always a String.
Review Comment:
I will open a follow on issue for this and remove the TODO before merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1917106492
##########
core/src/main/java/org/apache/accumulo/core/fate/AdminUtil.java:
##########
@@ -285,19 +286,19 @@ private void findLocks(ZooSession zk, final ServiceLockPath lockPath,
List<String> lockedIds = zr.getChildren(lockPath.toString());
for (String id : lockedIds) {
-
try {
-
FateLockPath fLockPath = FateLock.path(lockPath + "/" + id);
- List<String> lockNodes =
- FateLock.validateAndSort(fLockPath, zr.getChildren(fLockPath.toString()));
+ List<FateLock.FateLockNode> lockNodes =
+ FateLock.validateAndWarn(fLockPath, zr.getChildren(fLockPath.toString()));
+
+ lockNodes.sort(Comparator.comparingLong(ln -> ln.sequence));
Review Comment:
Updated to a sorted set in 3f25ce0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] speeds up fate lock acquisition [accumulo]
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on code in PR #5262:
URL: https://github.com/apache/accumulo/pull/5262#discussion_r1938321556
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -76,26 +77,10 @@ private FateLockEntry(LockType lockType, FateId fateId) {
this.fateId = Objects.requireNonNull(fateId);
}
- private FateLockEntry(byte[] entry) {
- if (entry == null || entry.length < 1) {
- throw new IllegalArgumentException();
- }
-
- int split = -1;
- for (int i = 0; i < entry.length; i++) {
- if (entry[i] == ':') {
- split = i;
- break;
- }
- }
-
- if (split == -1) {
- throw new IllegalArgumentException();
- }
-
- this.lockType = LockType.valueOf(new String(entry, 0, split, UTF_8));
- this.fateId =
- FateId.from(new String(Arrays.copyOfRange(entry, split + 1, entry.length), UTF_8));
+ private FateLockEntry(String entry) {
+ var fields = entry.split(":", 2);
Review Comment:
Any reason to verify the number of fields is 2? I suppose it will just fail a couple lines later if not correct.
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -106,14 +91,8 @@ public FateId getFateId() {
return fateId;
}
- public byte[] serialize() {
- byte[] typeBytes = lockType.name().getBytes(UTF_8);
- byte[] fateIdBytes = fateId.canonical().getBytes(UTF_8);
- byte[] result = new byte[fateIdBytes.length + 1 + typeBytes.length];
- System.arraycopy(typeBytes, 0, result, 0, typeBytes.length);
- result[typeBytes.length] = ':';
- System.arraycopy(fateIdBytes, 0, result, typeBytes.length + 1, fateIdBytes.length);
- return result;
+ public String serialize() {
+ return lockType.name() + ":" + fateId.canonical();
Review Comment:
This is SO much nicer, not having to do all that manual effort to copy bytes around is great.
##########
core/src/main/java/org/apache/accumulo/core/fate/zookeeper/FateLock.java:
##########
@@ -184,17 +214,11 @@ public SortedMap<Long,Supplier<FateLockEntry>> getEarlierEntries(long entry) {
}
for (String name : children) {
- // this try catch must be done inside the loop because some subset of the children may exist
- try {
- long order = Long.parseLong(name.substring(PREFIX.length()));
- if (order <= entry) {
- byte[] data = zoo.getData(path + "/" + name);
- // Use a supplier so we don't need to deserialize unless the calling code cares about
- // the value for that entry.
- result.put(order, Suppliers.memoize(() -> FateLockEntry.deserialize(data)));
- }
- } catch (KeeperException.NoNodeException ex) {
- // ignored
+ var parsed = new NodeName(name);
+ if (predicate.test(parsed.sequence, parsed.fateLockEntry)) {
+ // Use a supplier so we don't need to deserialize unless the calling code cares about
Review Comment:
You may want to move this comment to NodeName where the supplier is created now, but it doesn't matter too much either way probably
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org