You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jeremiah Jordan (Jira)" <ji...@apache.org> on 2023/05/24 18:55:00 UTC
[jira] [Commented] (CASSANDRA-18424) Implement graceful paging across tombstones with short-circuit on paging rather than throwing TombstoneOverwhelmingExceptions
[ https://issues.apache.org/jira/browse/CASSANDRA-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725920#comment-17725920 ]
Jeremiah Jordan commented on CASSANDRA-18424:
---------------------------------------------
[~jmckenzie] I would talk with [~jlewandowski] and the work he is doing for CASSANDRA-11745. There are some edge cases around stopping pagination early that had to be figured out, don't want you having to do that work twice.
> Implement graceful paging across tombstones with short-circuit on paging rather than throwing TombstoneOverwhelmingExceptions
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-18424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18424
> Project: Cassandra
> Issue Type: New Feature
> Components: Messaging/Client, Messaging/Internode
> Reporter: Josh McKenzie
> Assignee: Josh McKenzie
> Priority: Normal
>
> We implemented the hard stop with a {{TombstoneOverwhelmingException}} almost a decade ago since paging across many tombstones was the most common way for nodes to OOM as they iterated across all this data during queries and paging.
> With our current implementations and architecture / codebase, we should be able to combine the {{StoppingTransformation}} and existing {{clustering}} blob we pass back to clients to allow clients to optionally page across tombstones when using the async api via the driver and short-circuit a page when they hit the tombstone failure threshold rather than throwing a {{{}TombstoneOverwhelmingException{}}}. This would allow for more flexible data modeling on users' side as well as removing one of the fairly rough edges of our API's we're currently constrained by.
> Making sure this is correct will require extensive fuzz-testing of pagination; this should likely happen in the Harry project but we could also have a bespoke model / implementation in the C* codebase we rely on in the interim.
> Client warnings at the current default levels would remain; the gap between warn and "short-circuit pages" (100x ratio currently, 1000 vs. 100000) should be sufficient for clients to take action on their data models well before they hit this limit.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org