You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "belliottsmith (via GitHub)" <gi...@apache.org> on 2025/07/10 20:15:22 UTC

[PR] - Deduplicate ranges in WatermarkCollector [cassandra]

belliottsmith opened a new pull request, #4244:
URL: https://github.com/apache/cassandra/pull/4244

    - Collect ranges before submitting to TopologyManager to avoid quadratic complexity
   
   Thanks for sending a pull request! Here are some tips if you're new here:
    
    * Ensure you have added or run the [appropriate tests](https://cassandra.apache.org/_/development/testing.html) for your PR.
    * Be sure to keep the PR description updated to reflect all changes.
    * Write your PR title to summarize what this PR proposes.
    * If possible, provide a concise example to reproduce the issue for a faster review.
    * Read our [contributor guidelines](https://cassandra.apache.org/_/development/index.html)
    * If you're making a documentation change, see our [guide to documentation contribution](https://cassandra.apache.org/_/development/documentation.html)
    
   Commit messages should follow the following format:
   
   ```
   <One sentence description, usually Jira title or CHANGES.txt summary>
   
   <Optional lengthier description (context on patch)>
   
   patch by <Authors>; reviewed by <Reviewers> for CASSANDRA-#####
   
   Co-authored-by: Name1 <email1>
   Co-authored-by: Name2 <email2>
   
   ```
   
   The [Cassandra Jira](https://issues.apache.org/jira/projects/CASSANDRA/issues/)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] - Deduplicate ranges in WatermarkCollector [cassandra]

Posted by "belliottsmith (via GitHub)" <gi...@apache.org>.
belliottsmith merged PR #4244:
URL: https://github.com/apache/cassandra/pull/4244


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] - Deduplicate ranges in WatermarkCollector [cassandra]

Posted by "ifesdjeen (via GitHub)" <gi...@apache.org>.
ifesdjeen commented on code in PR #4244:
URL: https://github.com/apache/cassandra/pull/4244#discussion_r2201737063


##########
src/java/org/apache/cassandra/service/accord/WatermarkCollector.java:
##########
@@ -139,32 +149,49 @@ static void fetchAndReportWatermarksAsync(AccordConfigurationService configServi
 
                    Snapshot snapshot = m.payload;
                    long minEpoch = configService.minEpoch();
-                   for (Map.Entry<Range, Long> e : snapshot.closed.entrySet())
-                   {
-                       Ranges r = Ranges.of(e.getKey());
-                       configService.receiveClosed(r, e.getValue());
-                   }
-                   for (Map.Entry<Range, Long> e : snapshot.retired.entrySet())
-                   {
-                       Ranges r = Ranges.of(e.getKey());
-                       configService.receiveRetired(r, e.getValue());
-                   }
-                   for (Map.Entry<Integer, Long> e : snapshot.synced.entrySet())
+                   snapshot.retired.sort(sortByEpochThenRange);

Review Comment:
   This one will be sorted in forEachEpoch just below



##########
src/java/org/apache/cassandra/service/accord/WatermarkCollector.java:
##########
@@ -139,32 +149,49 @@ static void fetchAndReportWatermarksAsync(AccordConfigurationService configServi
 
                    Snapshot snapshot = m.payload;
                    long minEpoch = configService.minEpoch();
-                   for (Map.Entry<Range, Long> e : snapshot.closed.entrySet())
-                   {
-                       Ranges r = Ranges.of(e.getKey());
-                       configService.receiveClosed(r, e.getValue());
-                   }
-                   for (Map.Entry<Range, Long> e : snapshot.retired.entrySet())
-                   {
-                       Ranges r = Ranges.of(e.getKey());
-                       configService.receiveRetired(r, e.getValue());
-                   }
-                   for (Map.Entry<Integer, Long> e : snapshot.synced.entrySet())
+                   snapshot.retired.sort(sortByEpochThenRange);
+
+                   forEachEpoch(configService::receiveClosed, snapshot.closed);
+                   forEachEpoch(configService::receiveRetired, snapshot.retired);
+                   for (Map.Entry<Long, Long> e : snapshot.synced.entrySet())
                    {
-                       Node.Id node = new Node.Id(e.getKey());
+                       Node.Id node = new Node.Id(Ints.saturatedCast(e.getKey()));
                        for (long epoch = minEpoch; epoch <= e.getValue(); epoch++)
                            configService.receiveRemoteSyncComplete(node, epoch);
                    }
                });
     }
 
+    private static void forEachEpoch(BiConsumer<Ranges, Long> forEachEpoch, List<Map.Entry<Range, Long>> rangesAndEpochs)
+    {
+        if (rangesAndEpochs.isEmpty())
+            return;
+
+        rangesAndEpochs.sort(sortByEpochThenRange);
+        long collectingEpoch = rangesAndEpochs.get(0).getValue();
+        List<Range> ranges = new ArrayList<>();
+        for (Map.Entry<Range, Long> e : rangesAndEpochs)
+        {
+            Range range = e.getKey();
+            long epoch = e.getValue();
+            if (epoch != collectingEpoch)
+            {
+                forEachEpoch.accept(Ranges.of(ranges.toArray(Range[]::new)), collectingEpoch);
+                collectingEpoch = epoch;
+                ranges.clear();
+            }

Review Comment:
   Discussed on Slack, should we do 
   
   ```
               if (epoch != collectingEpoch)
               {
                   forEachEpoch.accept(Ranges.of(ranges.toArray(Range[]::new)), collectingEpoch);
                   collectingEpoch = epoch;
                   ranges.clear();
               }
               ranges.add(range);
   ```



##########
src/java/org/apache/cassandra/service/accord/WatermarkCollector.java:
##########
@@ -81,16 +89,15 @@ public class WatermarkCollector implements ConfigurationService.Listener
     @Override
     public void onRemoteSyncComplete(Node.Id node, long epoch)
     {
-        synced.compute(node.id, (k, prev) -> prev == null ? epoch : Long.max(prev, epoch));
+        synced.compute(node.id, (k, prev) -> prev == -1 ? epoch : Long.max(prev, epoch));

Review Comment:
   only realizing it now, but we may want to have a synchronized block around this one, or?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org