You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2022/10/11 11:02:10 UTC
[GitHub] [accumulo] keith-turner opened a new pull request, #3013: fixes user compaction stuck when producing no output
keith-turner opened a new pull request, #3013:
URL: https://github.com/apache/accumulo/pull/3013
This commit fixes a bug where :
* user compactions take multiple compaction steps (because the tablet has many files)
* the intermediate steps produce no output
When the above happened CompactableImpl and Tablet would disagree about what files there were. The Tablet code would ignore the empty file produced by an intermediate compaction. CompactableImpl would expect Tablet to know of this file. When this happened things would hang until a tserver was restarted.
Ran into this bug while continually running Bulk random walk test to reproduce #2667
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner merged pull request #3013: fixes user compaction stuck when producing no output
Posted by GitBox <gi...@apache.org>.
keith-turner merged PR #3013:
URL: https://github.com/apache/accumulo/pull/3013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on pull request #3013: fixes user compaction stuck when producing no output
Posted by GitBox <gi...@apache.org>.
dlmarion commented on PR #3013:
URL: https://github.com/apache/accumulo/pull/3013#issuecomment-1274529634
I don't know this area of the code well, did these changes fix the issue (did you test it already?). I kicked off a full IT build for this branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on pull request #3013: fixes user compaction stuck when producing no output
Posted by GitBox <gi...@apache.org>.
dlmarion commented on PR #3013:
URL: https://github.com/apache/accumulo/pull/3013#issuecomment-1274875796
Full IT build passed with no test failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on a diff in pull request #3013: fixes user compaction stuck when producing no output
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on code in PR #3013:
URL: https://github.com/apache/accumulo/pull/3013#discussion_r992672543
##########
test/src/main/java/org/apache/accumulo/test/functional/CompactionIT.java:
##########
@@ -494,6 +495,54 @@ public void testSuccessfulCompaction() throws Exception {
}
}
+ @Test
+ public void testMultiStepCompactionThatDeletesAll() throws Exception {
+
+ // There was a bug where user compactions would never complete when : the tablet had to be
+ // compacted in multiple passes AND the intermediate passes produced no output.
+
+ try (AccumuloClient c = Accumulo.newClient().from(getClientProps()).build()) {
+ final String tableName = getUniqueNames(1)[0];
+ c.tableOperations().create(tableName);
+ c.tableOperations().setProperty(tableName, Property.TABLE_MAJC_RATIO.getKey(), "100.0");
+
+ var beforeCount = countFiles(c);
+
+ try (var writer = c.createBatchWriter(tableName)) {
+ for (int i = 0; i < 60; i++) {
Review Comment:
A variable with a descriptive name could be added in place of `60` here to make it easier to tell what it represents when it is referenced throughout this test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on pull request #3013: fixes user compaction stuck when producing no output
Posted by GitBox <gi...@apache.org>.
keith-turner commented on PR #3013:
URL: https://github.com/apache/accumulo/pull/3013#issuecomment-1274571108
> I don't know this area of the code well, did these changes fix the issue (did you test it already?). I kicked off a full IT build for this branch.
The IT I added in the PR would cause the issue to happen. Without the fix the IT would get stuck. After adding the fix it would succeed. I just kicked off another round of Bulk RW test with this change, it was causing the walkers to get stuck. Going to see how that goes.
When it did get stuck would see lots of log messages like the following in the tserver logs over and over.
```
2022-10-10T16:27:10,577 [tablet.CompactableImpl] DEBUG: Selected files not in all files [hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jt.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jv.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010ml/I00010mm.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k2/I00010k4.rf, hdfs://localhost:8020/accumulo/tables/f/t-0000xmg/A0000zrs.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010ji/I00010jk.rf, hdfs://localhost:8020/accumulo/tables/f/t-0000xl5/A0000ztw.rf, hdfs://localhost:8020/accumulo/tables/f/t-0000zyu/C000119j.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jm.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010ll/I00010ln.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010ll/I00010lp.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jf/I00010jg.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k6/I00010k8.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010js.rf, hdfs://localhost:8020/ac
cumulo/tables/f/b-00010jj/I00010jq.rf, hdfs://localhost:8020/accumulo/tables/f/t-0000zyu/C00011ul.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k2/I00010k3.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jn.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k6/I00010k7.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jf/I00010jh.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k6/I00010k9.rf] [hdfs://localhost:8020/accumulo/tables/f/b-00010jf/I00010jg.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jf/I00010jh.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010ji/I00010jk.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jm.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jn.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jq.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010js.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jt.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010jj/I00010jv.rf, hdfs://local
host:8020/accumulo/tables/f/b-00010k2/I00010k3.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k2/I00010k4.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k6/I00010k7.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k6/I00010k8.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010k6/I00010k9.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010ll/I00010ln.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010ll/I00010lp.rf, hdfs://localhost:8020/accumulo/tables/f/b-00010ml/I00010mm.rf, hdfs://localhost:8020/accumulo/tables/f/t-0000xl5/A0000ztw.rf, hdfs://localhost:8020/accumulo/tables/f/t-0000xmg/A0000zrs.rf, hdfs://localhost:8020/accumulo/tables/f/t-0000zyu/C000119j.rf]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on pull request #3013: fixes user compaction stuck when producing no output
Posted by GitBox <gi...@apache.org>.
keith-turner commented on PR #3013:
URL: https://github.com/apache/accumulo/pull/3013#issuecomment-1274583797
I also ran all of the compaction ITs I could find and those passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org