[GitHub] [cassandra-accord] aweisberg commented on pull request #23: Introduce range transactions, including burn test support for for read-only transactions

[GitBox <gi...@apache.org>]

aweisberg commented on PR #23:
URL: https://github.com/apache/cassandra-accord/pull/23#issuecomment-1382429916

   I am a little mixed up on several things so to try and clarify. At each shard in terms of metadata about transaction execution over large numbers of transactions (as opposed to fetch this specific one) we need to materialize to schedule a transaction.
   
   1. We need the max conflict to response with an `executeAt` during `PreAccept`
   2. We need to calculate the dependency `TxnId`s that are < this `executeAt` during `PreAccept` or whenever trying to supply dependencies
   3. Calculate the timestamp to use as `nowInSeconds` for TTLs
   4. Calculate the timestamp to use when inserting as well as processing TTLs on the data we are inserting
   
   3 and 4 maybe we don't need to look at other transactions to calculate? I had asked some questions about that earlier.
   
   1 we can calculate pessimistically using an in-memory summary
   2 we might actually need to materialize a large number of IDs in a range transaction?
   
   We are going to materialize the entire set dependencies in memory at the coordinator to distribute to the replicas of each shard so there are some hard limits to how big those can practically get.
   
   So are we trying to be able to handle large amounts of dependencies better (which is memory bound) or are we trying to better handle smaller amounts that aren't in memory and need to be read off of disk?
   
   Even if we make this async isn't the shard blocked waiting on disk because of execution order constraints? Introducing a transaction with dependency data not in memory blocks subsequent transactions either way.
   
   Or is what we are optimizing for here keeping the shard thread unblocked so it can continue to work on things that are ready to work on and not blocked by execution order constraints?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org