You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2021/04/21 22:11:17 UTC
[GitHub] [accumulo] keith-turner opened a new issue #2028: Cancel external compactions when a user compaction is canceled
keith-turner opened a new issue #2028:
URL: https://github.com/apache/accumulo/issues/2028
Internal compaction use [this code][1] to determine if they should keep running a user compaction. Each user compaction has a 1 up per tablet id. There is also a per table cancel id in ZK, if its higher then the user compaction was canceled.
[1]:https://github.com/apache/accumulo/blob/42de4efab84b4dc0e4eda23c6d348be744941c30/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/CompactableImpl.java#L738-L757
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on issue #2028: Cancel external compactions when a user compaction is canceled
Posted by GitBox <gi...@apache.org>.
dlmarion commented on issue #2028:
URL: https://github.com/apache/accumulo/issues/2028#issuecomment-828401604
My guess is that it split the table and then another compaction kicked off. There were other places in the code where I changed `assertEquals(1, metrics.getStarted())` to `assertTrue(metrics.getStarted() > 0)`. I didn't necessarily care that it the value was `1`, just that it was non-zero.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on issue #2028: Cancel external compactions when a user compaction is canceled
Posted by GitBox <gi...@apache.org>.
dlmarion commented on issue #2028:
URL: https://github.com/apache/accumulo/issues/2028#issuecomment-828427882
Fixed in f8b906f, I think we can close this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner closed issue #2028: Cancel external compactions when a user compaction is canceled
Posted by GitBox <gi...@apache.org>.
keith-turner closed issue #2028:
URL: https://github.com/apache/accumulo/issues/2028
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on issue #2028: Cancel external compactions when a user compaction is canceled
Posted by GitBox <gi...@apache.org>.
dlmarion commented on issue #2028:
URL: https://github.com/apache/accumulo/issues/2028#issuecomment-828415033
I just ran into this. I think what's happening is that the Compactor fails because split is called and the compaction is cancelled. The tablet must have split and when the Compactor calls getNextJob() the Coordinator has another one ready to go. Here is the log from the Compactor:
```
2021-04-28T12:23:22,738 [coordinator.CompactionCoordinator] DEBUG: getCompactionJob called for queue DCQ1 by compactor ip-10-113-12-209.evoforge.org:9101
2021-04-28T12:23:22,739 [coordinator.CompactionCoordinator] DEBUG: Getting compaction for queue DCQ1 from tserver ip-10-113-12-209.evoforge.org:42012
2021-04-28T12:23:22,775 [coordinator.CompactionCoordinator] DEBUG: Returning external job ECID:36f38535-2a9b-450f-b5dd-f02ef27e8ddf to ip-10-113-12-209.evoforge.org:9101
2021-04-28T12:23:22,799 [coordinator.CompactionCoordinator] INFO : Compaction status update, id: ECID:36f38535-2a9b-450f-b5dd-f02ef27e8ddf, timestamp: 1619612602791, state: STARTED, message: Compaction started
2021-04-28T12:23:23,213 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:24,436 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:24,694 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:24,954 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:25,218 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:25,480 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:25,738 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:25,997 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:26,256 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:26,513 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:26,668 [server.GarbageCollectionLogger] DEBUG: gc G1 Young Generation=0.08(+0.01) secs G1 Old Generation=0.00(+0.00) secs freemem=207,560,904(-13,765,624) totalmem=268,435,456
2021-04-28T12:23:26,778 [test.TestCompactionCoordinator] DEBUG: Returning metrics: started: 1, running: 1, completed: 0, failed: 0
2021-04-28T12:23:26,832 [coordinator.CompactionCoordinator] INFO : Compaction status update, id: ECID:36f38535-2a9b-450f-b5dd-f02ef27e8ddf, timestamp: 1619612606831, state: CANCELLED, message: Compaction cancelled
2021-04-28T12:23:26,841 [coordinator.CompactionCoordinator] INFO : Compaction failed, id: ECID:36f38535-2a9b-450f-b5dd-f02ef27e8ddf
2021-04-28T12:23:26,872 [coordinator.CompactionCoordinator] DEBUG: getCompactionJob called for queue DCQ1 by compactor ip-10-113-12-209.evoforge.org:9101
2021-04-28T12:23:26,873 [coordinator.CompactionCoordinator] DEBUG: Getting compaction for queue DCQ1 from tserver ip-10-113-12-209.evoforge.org:42012
2021-04-28T12:23:26,884 [coordinator.CompactionCoordinator] DEBUG: Returning external job ECID:acd7e431-505f-4e7b-9315-108e11ccf620 to ip-10-113-12-209.evoforge.org:9101
2021-04-28T12:23:26,888 [coordinator.CompactionCoordinator] INFO : Compaction status update, id: ECID:acd7e431-505f-4e7b-9315-108e11ccf620, timestamp: 1619612606887, state: STARTED, message: Compaction started
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on issue #2028: Cancel external compactions when a user compaction is canceled
Posted by GitBox <gi...@apache.org>.
dlmarion commented on issue #2028:
URL: https://github.com/apache/accumulo/issues/2028#issuecomment-827087739
Wrote ExternalCompactionIT.testUserCompactionCancellation in commit b21af9fea21bff53f3779c057ddc5bc16c730ac9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion edited a comment on issue #2028: Cancel external compactions when a user compaction is canceled
Posted by GitBox <gi...@apache.org>.
dlmarion edited a comment on issue #2028:
URL: https://github.com/apache/accumulo/issues/2028#issuecomment-828415033
I just ran into this. I think what's happening is that the Compactor fails because split is called and the compaction is cancelled. The tablet must have split and when the Compactor calls getNextJob() the Coordinator has another one ready to go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #2028: Cancel external compactions when a user compaction is canceled
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2028:
URL: https://github.com/apache/accumulo/issues/2028#issuecomment-828043073
> Wrote ExternalCompactionIT.testUserCompactionCancellation in commit b21af9f
@dlmarion the test look great. I tweaked them a bit in 2c6095edd1592037b00a178e1e68b6b49001577c
When running them I saw the following error.
```
[INFO] Running org.apache.accumulo.test.ExternalCompactionIT
[ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 602.636 s <<< FAILURE! - in org.apache.accumulo.test.ExternalCompactionIT
[ERROR] org.apache.accumulo.test.ExternalCompactionIT.testSplitDuringExternalCompaction Time elapsed: 49.06 s <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<2>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:633)
at org.apache.accumulo.test.ExternalCompactionIT.testSplitDuringExternalCompaction(ExternalCompactionIT.java:227)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:834)
```
Line 227 is `assertEquals(1, metrics.getStarted());` any idea why that was off?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org