You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Vladislav Pyatkov (Jira)" <ji...@apache.org> on 2025/02/24 17:57:00 UTC
[jira] [Updated] (IGNITE-24614) Prevent throwing safe time advance exception on stop
[ https://issues.apache.org/jira/browse/IGNITE-24614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vladislav Pyatkov updated IGNITE-24614:
---------------------------------------
Description:
h3. Motivation
Node might throw {{TrackerClosedException}} an exception if the node was going to advise safe time but did not have time to make it:
{noformat}
[2025-02-24T15:45:26,966][ERROR][org.apache.ignite.internal.benchmark.MultiTableBenchmark.test-jmh-worker-4][ReplicaManager] Could not advance safe time for 429_part_22 to {}
java.util.concurrent.CompletionException: org.apache.ignite.internal.util.TrackerClosedException
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:636) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194) ~[?:?]
at org.apache.ignite.internal.util.PendingComparableValuesTracker.lambda$cleanupWaitersOnClose$2(PendingComparableValuesTracker.java:192) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
at org.apache.ignite.internal.util.PendingComparableValuesTracker.cleanupWaitersOnClose(PendingComparableValuesTracker.java:192) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.PendingComparableValuesTracker.close(PendingComparableValuesTracker.java:166) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.metastorage.server.time.ClusterTimeImpl.close(ClusterTimeImpl.java:142) ~[ignite-metastorage-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.lambda$closeAllManually$1(IgniteUtils.java:617) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:?]
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) ~[?:?]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
at org.apache.ignite.internal.util.IgniteUtils.closeAllManually(IgniteUtils.java:615) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.closeAllManually(IgniteUtils.java:649) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.metastorage.impl.MetaStorageManagerImpl.stopAsync(MetaStorageManagerImpl.java:772) ~[ignite-metastorage-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.lambda$stopAsync$6(IgniteUtils.java:1206) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[?:?]
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616) ~[?:?]
at org.apache.ignite.internal.util.IgniteUtils.stopAsync(IgniteUtils.java:1212) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.stopAsync(IgniteUtils.java:1254) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.LifecycleManager.initiateAllComponentsStop(LifecycleManager.java:178) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.LifecycleManager.stopNode(LifecycleManager.java:152) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteImpl.stopAsync(IgniteImpl.java:1610) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteServerImpl.doShutdownAsync(IgniteServerImpl.java:352) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteServerImpl.lambda$chainRestartOrShutdownAction$6(IgniteServerImpl.java:291) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2341) ~[?:?]
at org.apache.ignite.internal.app.IgniteServerImpl.chainRestartOrShutdownAction(IgniteServerImpl.java:291) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteServerImpl.shutdownAsync(IgniteServerImpl.java:328) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteServerImpl.shutdown(IgniteServerImpl.java:368) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.lambda$closeAll$0(IgniteUtils.java:570) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:?]
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) ~[?:?]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
at org.apache.ignite.internal.util.IgniteUtils.closeAll(IgniteUtils.java:568) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.benchmark.AbstractMultiNodeBenchmark.nodeTearDown(AbstractMultiNodeBenchmark.java:188) ~[integrationTest/:?]
at org.apache.ignite.internal.benchmark.jmh_generated.MultiTableBenchmark_test_jmhTest.test_Throughput(MultiTableBenchmark_test_jmhTest.java:103) ~[integrationTest/:?]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[?:?]
at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:527) ~[jmh-core-1.37.jar:?]
at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:504) ~[jmh-core-1.37.jar:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.apache.ignite.internal.util.TrackerClosedException
at org.apache.ignite.internal.util.PendingComparableValuesTracker.close(PendingComparableValuesTracker.java:164) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
... 59 more
{noformat}
This notification can be annoying in case the cluster has several tables with many partitions.
We are already ignoring the exceptions that are typical for stopping:
{code}
if (ex != null && !hasCauseOrSuppressed(ex, NodeStoppingException.class)
&& !hasCauseOrSuppressed(ex, CancellationException.class)) {
LOG.error("Could not advance safe time for {} to {}", ex, replica.groupId());
}
{code}
I believe it is a mistake that we are not ignoring TrackerClosedException also.
h3. Definition of done
The exception does not appear on stop.
was:
h3. Motivation
Node might throw {{TrackerClosedException}} an exception if the node was going to advise safe time but did not have time to make it:
{noformat}
[2025-02-24T15:45:26,966][ERROR][org.apache.ignite.internal.benchmark.MultiTableBenchmark.test-jmh-worker-4][ReplicaManager] Could not advance safe time for 429_part_22 to {}
java.util.concurrent.CompletionException: org.apache.ignite.internal.util.TrackerClosedException
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:636) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194) ~[?:?]
at org.apache.ignite.internal.util.PendingComparableValuesTracker.lambda$cleanupWaitersOnClose$2(PendingComparableValuesTracker.java:192) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
at org.apache.ignite.internal.util.PendingComparableValuesTracker.cleanupWaitersOnClose(PendingComparableValuesTracker.java:192) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.PendingComparableValuesTracker.close(PendingComparableValuesTracker.java:166) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.metastorage.server.time.ClusterTimeImpl.close(ClusterTimeImpl.java:142) ~[ignite-metastorage-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.lambda$closeAllManually$1(IgniteUtils.java:617) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:?]
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) ~[?:?]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
at org.apache.ignite.internal.util.IgniteUtils.closeAllManually(IgniteUtils.java:615) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.closeAllManually(IgniteUtils.java:649) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.metastorage.impl.MetaStorageManagerImpl.stopAsync(MetaStorageManagerImpl.java:772) ~[ignite-metastorage-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.lambda$stopAsync$6(IgniteUtils.java:1206) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[?:?]
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616) ~[?:?]
at org.apache.ignite.internal.util.IgniteUtils.stopAsync(IgniteUtils.java:1212) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.stopAsync(IgniteUtils.java:1254) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.LifecycleManager.initiateAllComponentsStop(LifecycleManager.java:178) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.LifecycleManager.stopNode(LifecycleManager.java:152) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteImpl.stopAsync(IgniteImpl.java:1610) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteServerImpl.doShutdownAsync(IgniteServerImpl.java:352) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteServerImpl.lambda$chainRestartOrShutdownAction$6(IgniteServerImpl.java:291) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2341) ~[?:?]
at org.apache.ignite.internal.app.IgniteServerImpl.chainRestartOrShutdownAction(IgniteServerImpl.java:291) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteServerImpl.shutdownAsync(IgniteServerImpl.java:328) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.app.IgniteServerImpl.shutdown(IgniteServerImpl.java:368) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.util.IgniteUtils.lambda$closeAll$0(IgniteUtils.java:570) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:?]
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) ~[?:?]
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) ~[?:?]
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
at org.apache.ignite.internal.util.IgniteUtils.closeAll(IgniteUtils.java:568) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
at org.apache.ignite.internal.benchmark.AbstractMultiNodeBenchmark.nodeTearDown(AbstractMultiNodeBenchmark.java:188) ~[integrationTest/:?]
at org.apache.ignite.internal.benchmark.jmh_generated.MultiTableBenchmark_test_jmhTest.test_Throughput(MultiTableBenchmark_test_jmhTest.java:103) ~[integrationTest/:?]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[?:?]
at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:527) ~[jmh-core-1.37.jar:?]
at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:504) ~[jmh-core-1.37.jar:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.apache.ignite.internal.util.TrackerClosedException
at org.apache.ignite.internal.util.PendingComparableValuesTracker.close(PendingComparableValuesTracker.java:164) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
... 59 more
{noformat}
This notification can be annoying in case the cluster has several tables with many partitions.
> Prevent throwing safe time advance exception on stop
> ----------------------------------------------------
>
> Key: IGNITE-24614
> URL: https://issues.apache.org/jira/browse/IGNITE-24614
> Project: Ignite
> Issue Type: Bug
> Reporter: Vladislav Pyatkov
> Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> Node might throw {{TrackerClosedException}} an exception if the node was going to advise safe time but did not have time to make it:
> {noformat}
> [2025-02-24T15:45:26,966][ERROR][org.apache.ignite.internal.benchmark.MultiTableBenchmark.test-jmh-worker-4][ReplicaManager] Could not advance safe time for 429_part_22 to {}
> java.util.concurrent.CompletionException: org.apache.ignite.internal.util.TrackerClosedException
> at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) ~[?:?]
> at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347) ~[?:?]
> at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:636) ~[?:?]
> at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
> at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194) ~[?:?]
> at org.apache.ignite.internal.util.PendingComparableValuesTracker.lambda$cleanupWaitersOnClose$2(PendingComparableValuesTracker.java:192) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
> at org.apache.ignite.internal.util.PendingComparableValuesTracker.cleanupWaitersOnClose(PendingComparableValuesTracker.java:192) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.util.PendingComparableValuesTracker.close(PendingComparableValuesTracker.java:166) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.metastorage.server.time.ClusterTimeImpl.close(ClusterTimeImpl.java:142) ~[ignite-metastorage-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.util.IgniteUtils.lambda$closeAllManually$1(IgniteUtils.java:617) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:?]
> at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
> at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
> at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) ~[?:?]
> at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
> at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
> at org.apache.ignite.internal.util.IgniteUtils.closeAllManually(IgniteUtils.java:615) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.util.IgniteUtils.closeAllManually(IgniteUtils.java:649) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.metastorage.impl.MetaStorageManagerImpl.stopAsync(MetaStorageManagerImpl.java:772) ~[ignite-metastorage-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.util.IgniteUtils.lambda$stopAsync$6(IgniteUtils.java:1206) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
> at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
> at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[?:?]
> at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616) ~[?:?]
> at org.apache.ignite.internal.util.IgniteUtils.stopAsync(IgniteUtils.java:1212) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.util.IgniteUtils.stopAsync(IgniteUtils.java:1254) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.app.LifecycleManager.initiateAllComponentsStop(LifecycleManager.java:178) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.app.LifecycleManager.stopNode(LifecycleManager.java:152) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.app.IgniteImpl.stopAsync(IgniteImpl.java:1610) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.app.IgniteServerImpl.doShutdownAsync(IgniteServerImpl.java:352) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.app.IgniteServerImpl.lambda$chainRestartOrShutdownAction$6(IgniteServerImpl.java:291) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
> at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187) ~[?:?]
> at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2341) ~[?:?]
> at org.apache.ignite.internal.app.IgniteServerImpl.chainRestartOrShutdownAction(IgniteServerImpl.java:291) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.app.IgniteServerImpl.shutdownAsync(IgniteServerImpl.java:328) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.app.IgniteServerImpl.shutdown(IgniteServerImpl.java:368) ~[ignite-runner-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.util.IgniteUtils.lambda$closeAll$0(IgniteUtils.java:570) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) ~[?:?]
> at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
> at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
> at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
> at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) ~[?:?]
> at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) ~[?:?]
> at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
> at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
> at org.apache.ignite.internal.util.IgniteUtils.closeAll(IgniteUtils.java:568) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> at org.apache.ignite.internal.benchmark.AbstractMultiNodeBenchmark.nodeTearDown(AbstractMultiNodeBenchmark.java:188) ~[integrationTest/:?]
> at org.apache.ignite.internal.benchmark.jmh_generated.MultiTableBenchmark_test_jmhTest.test_Throughput(MultiTableBenchmark_test_jmhTest.java:103) ~[integrationTest/:?]
> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[?:?]
> at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[?:?]
> at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:527) ~[jmh-core-1.37.jar:?]
> at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:504) ~[jmh-core-1.37.jar:?]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
> at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
> Caused by: org.apache.ignite.internal.util.TrackerClosedException
> at org.apache.ignite.internal.util.PendingComparableValuesTracker.close(PendingComparableValuesTracker.java:164) ~[ignite-core-3.1.0-SNAPSHOT.jar:?]
> ... 59 more
> {noformat}
> This notification can be annoying in case the cluster has several tables with many partitions.
> We are already ignoring the exceptions that are typical for stopping:
> {code}
> if (ex != null && !hasCauseOrSuppressed(ex, NodeStoppingException.class)
> && !hasCauseOrSuppressed(ex, CancellationException.class)) {
> LOG.error("Could not advance safe time for {} to {}", ex, replica.groupId());
> }
> {code}
> I believe it is a mistake that we are not ignoring TrackerClosedException also.
> h3. Definition of done
> The exception does not appear on stop.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)