[jira] [Commented] (CASSANDRA-10011) I want to develop transactions for Cassandra and I want your feedback

["Philip Thompson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-10011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660814#comment-14660814 ] 

Philip Thompson commented on CASSANDRA-10011:
---------------------------------------------

This might be better to discuss on the developer's mailing list.

> I want to develop transactions for Cassandra and I want your feedback
> ---------------------------------------------------------------------
>
>                 Key: CASSANDRA-10011
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10011
>             Project: Cassandra
>          Issue Type: New Feature
>            Reporter: Marek Lewandowski
>              Labels: features
>
> Hello everyone,
> *TL;DR;* I want to develop transactions (similar to those relational ones) for Cassandra, I have some ideas and I'd like to hear your feedback.
> *Long story short:* I want to develop prototype of solution that features transactions spanning multiple Cassandra partitions resembling those in relational databases. I understand that in Cassandra's world such transactions will be quite a bit different than in relational db world. That prototype or research if you will, is subject of my master's thesis. 
> It's been some time since I've been dancing around that subject and postponing it, but what I understood in the process is that I cannot do anything useful being hidden from community's feedback. I want you to provide feedback on my ideas. You also have a power of changing it completely because first and foremost _I want to develop something actually useful, not only get my master's degree._
> To not scare anyone with huge wall of text, for now I'll just post very brief description of my ideas and longer block of text describing how I can see committing and rolling back data. I have more detailed descriptions prepared already, but I don't want to share 4 pages of text in one go.
> Scope of the project (assuming I'll do it alone) is an experiment, not production ready solution.
> Such experiment can use any tools possible to actually perform it.
> Baseline for any idea is:
> - asynchronous execution - think Akka and actors with non blocking execution and message passing 
> - doesn't have to be completely transparent for end user - solution may enforce certain workflow of interacting with it and/or introduce new concepts (like akka messages instead of CQL and binary protocol).
> So far after reading a lot and thinking even more I have two ideas that I'd like to share.
> h3. Ideas are (with brief description, details will come later):
> h4. Idea 1: Event streaming 
> - Imagine that every modiifcation is represented by an _Event_. 
> - Imagine you can group these events into _Event Groups_. 
> - Imagine that such groups are time series data
> - Imagine you can read such groups as a stream of events (think reactive stream)
> Idea is that: you don't need to lock data when you are sure there is no one else to compete with.
> There is 1 guy called _Cursor_ that reads Event Stream and executes Event Groups one by one advacing its position on the stream when Event Group has been executed.
> Seems like a system where you have only 1 transaction at any given time, but there are many areas to optimize that and to allow more than that. However I'll stop here.
> h4. Idea 2: Locking data 
> - uses additional tables to acquire read/write locks
> - seperate tables to append modifications - as in "Rollback: Appending to seperate table."  
> - supports different isolation levels. 
> - more traditional approach, kind of translation of traditional locking to cassandra reality.
> -------------------------------------------------------------------------------------------
> Common part of two is approach to doing commit and rollback.
> h3. Doing Rollback and commit
> I have two ideas for rollback. I like 2nd one more, because it is simpler and potentially faster.
> h4. Rollback: Query rewriting
> It modifies original data, but before that it copies original data so that state can be restored. Then when failure is detected, modification query can be rewritten so that original data can be restored.
> Query rewriting seems like a complex functionality. I tried few simple and a little bit more complex statements and in general for basic stuff algorithm is not that complicated, but to support everything CQL has to offer it might be hard. 
> Still such transactional system might have some restrictions over CQL statements used, because first of all when someone wants to have these transactions they already want something non standard.
> I will skip details of that approach for now.
> h4. Rollback: Appending to seperate table.
> Image we have table A that we want to have transactions on.
> This requires another table A_tx which has same schema as A, but has *1 more clustering column* and few new columns. A_tx will be additionally clustered by transaction id. 
> New columns are:
> - {{is_committed boolean}}
> - {{is_rolledback boolean}}
> - {{is_applied boolean}}
> General idea is: 
> 	1. During transaction append changes to XXX_tx tables.
> 	2. For rollback: nothing needs to be done (besides cleaning XXX_tx tables of useless data scheduled for rollback)
> 	3. For commit: rows in each XXX_tx are marked as committed. This can be done using BATCH update so that all rows affected by transactions are committed. These changes will be eventually merged back into original row.
> Committed changes are visible during query, because query has to select from 2 tables. If you query for XXX table then you have to query that table, but also XXX_TX and get all committed data, merge result and return that to client.
> Here committed data eventually lands into proper row - during read as background process for example (this is this is_applied column) results are merged and inserted into original row, plus additionally modifications can be marked as _applied_. 
> Uncommitted data can also be eventually cleaned up.
> *Important note:* since partitioning key stays the same for {{XXX}} table and {{XXX_TX}} table, data will reside on same nodes so that queries and application of data can be done locally.
> h3. What happens next
> Assuming I get any feedback I'll post more detailed descriptions of two approaches.
> I would love to hear your feedback on whole subject. Just to begin discussion and pick your interest.
> What you think about having more heavy transactions? 
> Does this experiment has sense at all?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)