You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "David Capwell (Jira)" <ji...@apache.org> on 2023/04/20 23:06:00 UTC
[jira] [Updated] (CASSANDRA-18471) CEP-15 Accord: BurnTest fails with Received replies from a node that must have known the route, but that did not include it
[ https://issues.apache.org/jira/browse/CASSANDRA-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Capwell updated CASSANDRA-18471:
--------------------------------------
Bug Category: Parent values: Availability(12983)Level 1 values: Response Crash(12991)
Complexity: Normal
Discovered By: Fuzz Test
Fix Version/s: 5.x
Severity: Normal
Status: Open (was: Triage Needed)
> CEP-15 Accord: BurnTest fails with Received replies from a node that must have known the route, but that did not include it
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-18471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18471
> Project: Cassandra
> Issue Type: Bug
> Components: Accord
> Reporter: David Capwell
> Priority: Normal
> Fix For: 5.x
>
>
> While working on CASSANDRA-18451 I hit the following failure
> {code}
> Failed on seed -5929214838499924343
> accord.burn.SimulationException: Failed on seed -5929214838499924343
> Caused by: java.lang.AssertionError: Unexpected exception encountered
> at accord.impl.basic.PropagatingPendingQueue.poll(PropagatingPendingQueue.java:73)
> at accord.impl.basic.Cluster.processPending(Cluster.java:179)
> at accord.impl.basic.Cluster.run(Cluster.java:296)
> at accord.burn.BurnTest.burn(BurnTest.java:309)
> at accord.burn.BurnTest.run(BurnTest.java:386)
> at accord.burn.BurnTest.testOne(BurnTest.java:372)
> Suppressed: java.lang.IllegalStateException: Received replies from a node that must have known the route, but that did not include it
> at accord.coordinate.Invalidate.invalidate(Invalidate.java:204)
> at accord.coordinate.Invalidate.handle(Invalidate.java:131)
> at accord.coordinate.Invalidate.onSuccess(Invalidate.java:105)
> at accord.coordinate.Invalidate.onSuccess(Invalidate.java:51)
> at accord.impl.basic.Cluster.lambda$processNext$1(Cluster.java:209)
> at accord.impl.basic.Cluster.now(Cluster.java:260)
> at accord.impl.basic.Cluster.processNext(Cluster.java:206)
> at accord.impl.basic.Cluster.processPending(Cluster.java:183)
> {code}
> In a debugger was able to figure out the state and create a unit test to hit the same situation
> {code}
> class InvalidateTest
> {
> @Test
> void test() throws ExecutionException
> {
> try (MockCluster cluster = MockCluster.builder().replication(2).nodes(2).build())
> {
> Node n1 = cluster.get(1);
> Node n2 = cluster.get(2);
> RoutingKey n1RoutingKey = n1.topology().current().get(0).range.end();
> IntKey.Raw n1key = IntKey.key(((IntKey.Routing) n1RoutingKey).key);
> RoutingKey n2RoutingKey = n1.topology().current().get(1).range.end();
> IntKey.Raw n2key = IntKey.key(((IntKey.Routing) n2RoutingKey).key);
> Keys keys = Keys.of(n1key, n2key);
> Node coordinator = n1;
> TxnId txnId = coordinator.nextTxnId(Txn.Kind.Read, Routable.Domain.Key);
> Txn txn = readOnly(keys);
> AsyncChains.getUninterruptibly(n2.commandStores().unsafeForKey(n2key).execute(PreLoadContext.contextFor(txnId, keys), store -> {
> Ranges ranges = store.ranges().currentRanges();
> PartialTxn partial = txn.slice(ranges, true);
> FullKeyRoute route = keys.toRoute(n2RoutingKey);
> // RoutingKey progressKey = n2RoutingKey.toUnseekable(); // if this is non-null this passes
> RoutingKey progressKey = null;
> CheckedCommands.preaccept(store, txnId, partial, route, progressKey);
> CheckedCommands.accept(store, txnId, Ballot.ZERO, route.slice(ranges), partial.keys().slice(ranges), progressKey, txnId, PartialDeps.builder(ranges).build());
> }));
> AsyncChains.getUninterruptibly(new AsyncChains.Head<Outcome>() {
> @Override
> protected void start(BiConsumer<? super Outcome, Throwable> callback) {
> Invalidate.invalidate(coordinator, txnId, keys.toUnseekables(), callback);
> }
> });
> }
> }
> private static Txn readOnly(Seekables<?, ?> keys)
> {
> Read read = MockStore.read(keys);
> Query query = Mockito.mock(Query.class);
> return new Txn.InMemory(keys, read, query);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org