You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "dcapwell (via GitHub)" <gi...@apache.org> on 2024/03/04 20:02:43 UTC

[PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

dcapwell opened a new pull request, #3157:
URL: https://github.com/apache/cassandra/pull/3157

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "belliottsmith (via GitHub)" <gi...@apache.org>.
belliottsmith commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1522902019


##########
src/java/org/apache/cassandra/service/accord/AccordKeyspace.java:
##########
@@ -191,6 +203,15 @@ public class AccordKeyspace
 
     private static final ClusteringIndexFilter FULL_PARTITION = new ClusteringIndexSliceFilter(Slices.ALL, false);
 
+    //TODO (now, performance): should this be partitioner rather than TableId?  As of this patch distributed tables should only have 1 partitioner...
+    private static final ConcurrentMap<TableId, AccordRoutingKeyByteSource.Serializer> TABLE_SERIALIZERS = new ConcurrentHashMap<>();

Review Comment:
   this is primarily a read-cache, so it might be nice to optimise for this and not have the weight and cost of a concurrent hash map on read (e.g. CoW the whole map but use an Object2Object open hash map). 
   
   We should also avoid infinite state growth under schema changes, so it should either periodically be cleared, or have entries removed when tables are dropped, or on update should clear any stale entries (if we're doing CoW this is prob my favoured approach).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511820249


##########
test/unit/org/apache/cassandra/repair/FuzzTestBase.java:
##########
@@ -540,27 +528,11 @@ private static RepairOption repairOption(RandomSource rs, Cluster.Node coordinat
     {
         RepairType type = repairTypeGen.next(rs);
         PreviewType previewType = previewTypeGen.next(rs);
-        boolean accordRepair = type == RepairType.FULL && previewType == PreviewType.NONE ? rs.nextBoolean() : false;
         List<String> args = new ArrayList<>();
         args.add(ks);
         List<String> tables = tablesGen.next(rs);
         args.addAll(tables);
-        if (accordRepair)

Review Comment:
   this was adding non-deterministic behavior to repair tests.  I spoke with Ariel about this and also pointed out that the new accord logic is blocking (repair expects async)... I didn't want to tackle fixing this logic so just removing this for now to keep CI stable...
   
   we will need to fix the accord repair logic as it can cause perf regressions for repair



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511822806


##########
test/unit/org/apache/cassandra/service/accord/AccordKeyspaceTest.java:
##########
@@ -97,4 +134,157 @@ public void serde()
         Command loaded = AccordKeyspace.loadCommand(store, id);
         Assertions.assertThat(loaded).isEqualTo(committed);
     }
+
+    @Test
+    public void findOverlappingKeys()
+    {
+        var tableIdGen = fromQT(CassandraGenerators.TABLE_ID_GEN);
+        var partitionGen = fromQT(CassandraGenerators.partitioners());
+
+        var sstableFormats = DatabaseDescriptor.getSSTableFormats();
+        List<String> sstableFormatNames = new ArrayList<>(sstableFormats.keySet());
+        sstableFormatNames.sort(Comparator.naturalOrder());
+
+        List<String> memtableFormats = MemtableParams.knownDefinitions().stream()
+                                                     .filter(name -> !name.startsWith("test_") && !name.equals("default"))
+                                                     .sorted()
+                                                     .collect(Collectors.toList());

Review Comment:
   I honestly added this because I found data loss once we hit SSTables... turns out internal write logic doesn't do enough validation (as we do this in CQL) so we can write to Memtable, but this gets filtered out when we write to SSTables...  Testing the different formats only came in after to help me debug this issue... and figured its nice to do this anyways =)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511779832


##########
src/java/org/apache/cassandra/service/accord/AccordStateCache.java:
##########
@@ -295,6 +343,11 @@ public S acquireOrInitialize(K key, Function<K, V> valueFactory)
                 node = nodeFactory.create(key, index);
                 node.initialize(valueFactory.apply(key));
                 cache.put(key, node);
+                if (listeners != null)
+                {
+                    AccordCachingState<K, V> finalNode = node;

Review Comment:
   hopefully java 17 removes the requirement to copy this... =(



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511796144


##########
src/java/org/apache/cassandra/service/accord/serializers/AccordRoutingKeyByteSource.java:
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.service.accord.serializers;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.UUID;
+import java.util.function.Function;
+
+import org.apache.cassandra.db.marshal.ByteArrayAccessor;
+import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.ValueAccessor;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.schema.TableId;
+import org.apache.cassandra.service.accord.api.AccordRoutingKey;
+import org.apache.cassandra.utils.ByteArrayUtil;
+import org.apache.cassandra.utils.bytecomparable.ByteComparable;
+import org.apache.cassandra.utils.bytecomparable.ByteSource;
+import org.apache.cassandra.utils.bytecomparable.ByteSourceInverse;
+
+import static org.apache.cassandra.service.accord.api.AccordRoutingKey.RoutingKeyKind.SENTINEL;
+
+public class AccordRoutingKeyByteSource

Review Comment:
   names are hard...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511772426


##########
src/java/org/apache/cassandra/schema/TableId.java:
##########
@@ -50,7 +51,8 @@
 public class TableId implements Comparable<TableId>
 {
     public static final long MAGIC = 1956074401491665062L;
-    // TODO: should this be a TimeUUID?

Review Comment:
   FYI TCM moved away from time uuid so ids are deterministic now... so this comment doesn't make sense with TCM



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511826671


##########
test/unit/org/apache/cassandra/service/accord/AccordKeyspaceTest.java:
##########
@@ -97,4 +134,157 @@ public void serde()
         Command loaded = AccordKeyspace.loadCommand(store, id);
         Assertions.assertThat(loaded).isEqualTo(committed);
     }
+
+    @Test
+    public void findOverlappingKeys()
+    {
+        var tableIdGen = fromQT(CassandraGenerators.TABLE_ID_GEN);
+        var partitionGen = fromQT(CassandraGenerators.partitioners());
+
+        var sstableFormats = DatabaseDescriptor.getSSTableFormats();
+        List<String> sstableFormatNames = new ArrayList<>(sstableFormats.keySet());
+        sstableFormatNames.sort(Comparator.naturalOrder());
+
+        List<String> memtableFormats = MemtableParams.knownDefinitions().stream()
+                                                     .filter(name -> !name.startsWith("test_") && !name.equals("default"))
+                                                     .sorted()
+                                                     .collect(Collectors.toList());
+
+        qt().check(rs -> {
+            AccordKeyspace.unsafeClear();
+            // control SSTable format
+            setSelectedSSTableFormat(sstableFormats.get(rs.pick(sstableFormatNames)));
+            // control memtable format
+            setMemtable(ACCORD_KEYSPACE_NAME, "commands_for_key", rs.pick(memtableFormats));
+
+            // define the tables w/ partitioners for the test
+            // this uses the ability to override the SchemaProvider for the keyspace and only defines the single API call expected: getTablePartitioner
+            TreeMap<TableId, IPartitioner> tables = new TreeMap<>();
+            int numTables = rs.nextInt(1, 3);
+            for (int i = 0; i < numTables; i++)
+            {
+                var tableId = tableIdGen.next(rs);
+                while (tables.containsKey(tableId))
+                    tableId = tableIdGen.next(rs);
+                tables.put(tableId, partitionGen.next(rs));
+            }
+            SchemaProvider schema = Mockito.mock(SchemaProvider.class);
+            Mockito.when(schema.getTablePartitioner(Mockito.any())).thenAnswer((Answer<IPartitioner>) invocationOnMock -> tables.get(invocationOnMock.getArgument(0)));
+            AccordKeyspace.unsafeSetSchema(schema);
+
+            int numStores = rs.nextInt(1, 3);
+
+            // The model of the DB
+            TreeMap<Integer, SortedSet<PartitionKey>> storesToKeys = new TreeMap<>();
+            // write to the table and the model
+            for (int i = 0, numKeys = rs.nextInt(10, 20); i < numKeys; i++)
+            {
+                int store = rs.nextInt(0, numStores);
+                var keys = storesToKeys.computeIfAbsent(store, ignore -> new TreeSet<>());
+                PartitionKey pk = null;
+                // LocalPartitioner may have a type with a very small domain (boolean, vector<boolean, 1>, etc.), so need to bound the attempts
+                // else this will loop forever...
+                for (int attempt = 0; attempt < 10; attempt++)
+                {
+                    TableId tableId = rs.pick(tables.keySet());
+                    IPartitioner partitioner = tables.get(tableId);
+                    ByteBuffer data = !(partitioner instanceof LocalPartitioner) ? Int32Type.instance.decompose(rs.nextInt())
+                                                                                 : fromQT(getTypeSupport(partitioner.getTokenValidator()).bytesGen()).next(rs);
+                    PartitionKey key = new PartitionKey(tableId, tables.get(tableId).decorateKey(data));
+                    if (keys.add(key))
+                    {
+                        pk = key;
+                        break;
+                    }
+                }
+                if (pk != null)
+                {
+                    try
+                    {
+                        // using Mutation directly (what we do in Accord) can break when user data is too large; leading to data loss
+                        // The memtable will allow the write, but it will be dropped when writing to the SSTable...
+                        //TODO (now, correctness): since we store the user token + user key, if a key is close to the PK limits then we could tip over and loose our CFK
+//                        new Mutation(AccordKeyspace.getCommandsForKeyPartitionUpdate(store, pk, 42, ByteBufferUtil.EMPTY_BYTE_BUFFER)).apply();

Review Comment:
   this is the logic CFK uses, but this can cause problems down the road.  For this test these keys are filtered out and CQL is used directly to avoid these issues...
   
   There is a task to fix CFK to not have size limits that different than normal CQL



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "belliottsmith (via GitHub)" <gi...@apache.org>.
belliottsmith commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1522902019


##########
src/java/org/apache/cassandra/service/accord/AccordKeyspace.java:
##########
@@ -191,6 +203,15 @@ public class AccordKeyspace
 
     private static final ClusteringIndexFilter FULL_PARTITION = new ClusteringIndexSliceFilter(Slices.ALL, false);
 
+    //TODO (now, performance): should this be partitioner rather than TableId?  As of this patch distributed tables should only have 1 partitioner...
+    private static final ConcurrentMap<TableId, AccordRoutingKeyByteSource.Serializer> TABLE_SERIALIZERS = new ConcurrentHashMap<>();

Review Comment:
   this is primarily a read-cache, so it might be nice to optimise for this and not have the weight and cost of a concurrent hash map on read (e.g. CoW the whole map but use an Object2Object open hash map). This should also avoid infinite state growth under schema changes, so should either periodically be cleared, or should have entries removed when tables are dropped, or on update should clear any stale entries.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511776567


##########
src/java/org/apache/cassandra/service/accord/AccordMessageSink.java:
##########
@@ -222,7 +222,7 @@ public void send(Node.Id to, Request request)
         Preconditions.checkNotNull(verb, "Verb is null for type %s", request.type());
         Message<Request> message = Message.out(verb, request);
         InetAddressAndPort endpoint = endpointMapper.mappedEndpoint(to);
-        logger.debug("Sending {} {} to {}", verb, message.payload, endpoint);

Review Comment:
   this is super dense / verbose and slows things down and makes real bugs hard to find in the logs... 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511778513


##########
src/java/org/apache/cassandra/service/accord/AccordSafeCommandsForRanges.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.service.accord;
+
+import java.util.NavigableMap;
+import java.util.Objects;
+
+import accord.primitives.Range;
+import accord.primitives.Ranges;
+import accord.primitives.TxnId;
+import accord.utils.async.AsyncChains;
+import accord.utils.async.AsyncResult;
+import org.apache.cassandra.utils.Pair;
+
+public class AccordSafeCommandsForRanges implements AccordSafeState<Range, CommandsForRanges>

Review Comment:
   since `CommandsForRanges` is immutable and we don't support adding to it atm, this class isn't 100% needed... but figure if we did want to think about caching we would need this, so better to plumb it in now



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "smiklosovic (via GitHub)" <gi...@apache.org>.
smiklosovic closed pull request #3157: CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges
URL: https://github.com/apache/cassandra/pull/3157


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] CASSANDRA-19355: Accord: PreLoadContext must properly and consistently support ranges [cassandra]

Posted by "dcapwell (via GitHub)" <gi...@apache.org>.
dcapwell commented on code in PR #3157:
URL: https://github.com/apache/cassandra/pull/3157#discussion_r1511779187


##########
src/java/org/apache/cassandra/service/accord/AccordStateCache.java:
##########
@@ -278,13 +303,36 @@ public Instance(
             this.nodeFactory = nodeFactory;
         }
 
+        public void register(Listener<K, V> l)
+        {
+            if (listeners == null)
+                listeners = new ArrayList<>();
+            listeners.add(l);
+        }
+
+        public void unregister(Listener<K, V> l)
+        {
+            if (listeners == null)
+                throw new AssertionError("No listeners exist");
+            if (!listeners.remove(l))
+                throw new AssertionError("Listener was not registered");
+            if (listeners.isEmpty())
+                listeners = null;
+        }
+
         public Stream<AccordCachingState<K, V>> stream()
         {
             return cache.entrySet().stream()
-                        .filter(e -> keyClass.isAssignableFrom(e.getKey().getClass()))

Review Comment:
   we were matching multiple instances due to CFK stuff having the same type key...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org