You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by AJ <aj...@dude.podzone.net> on 2011/06/21 19:30:18 UTC
Storing Accounting Data
Is C* suitable for storing customer account (financial) data, as well as
billing, payroll, etc? This is a new company so migration is not an
issue... starting from scratch.
Thanks!
Re: Storing Accounting Data
Posted by Edward Capriolo <ed...@gmail.com>.
On Wed, Jun 22, 2011 at 10:19 AM, Oleg Anastastasyev <ol...@gmail.com>wrote:
> >
> > Is C* suitable for storing customer account (financial) data, as well as
> > billing, payroll, etc? This is a new company so migration is not an
> > issue... starting from scratch.
>
> If you need only store them - then yes, but if you require transactions
> spanning
> multiple rows or column families, which i believe will be main
> functionality
> here - then definitely no, because cassandra has no ACID, no transactions
> spanning multiple rows and no ability to rollback.
>
>
>
>
>
>
Well be careful, a financial transaction does not really have much to do
with a database transaction. For example, if I write a check that turn out
to be phoney three days later, it is not as if a database connection was
open for three days waiting for someone to type commit or rollback! So while
ACID, transactions, and rollbacks could be useful in this type of
applications, in a real world scenario most still need a complex application
layer.
Re: Storing Accounting Data
Posted by Sasha Dolgy <sd...@gmail.com>.
but you can store the -details- of a transaction as json data and do
some sanity checks to validate that the data you currently have stored
aligns with the recorded transactions. maybe a batch job run every 24
hours ...
On Wed, Jun 22, 2011 at 4:19 PM, Oleg Anastastasyev <ol...@gmail.com> wrote:
>>
>> Is C* suitable for storing customer account (financial) data, as well as
>> billing, payroll, etc? This is a new company so migration is not an
>> issue... starting from scratch.
>
> If you need only store them - then yes, but if you require transactions spanning
> multiple rows or column families, which i believe will be main functionality
> here - then definitely no, because cassandra has no ACID, no transactions
> spanning multiple rows and no ability to rollback.
Re: Storing Accounting Data
Posted by Oleg Anastastasyev <ol...@gmail.com>.
>
> Is C* suitable for storing customer account (financial) data, as well as
> billing, payroll, etc? This is a new company so migration is not an
> issue... starting from scratch.
If you need only store them - then yes, but if you require transactions spanning
multiple rows or column families, which i believe will be main functionality
here - then definitely no, because cassandra has no ACID, no transactions
spanning multiple rows and no ability to rollback.
Re: Storing Accounting Data
Posted by AJ <aj...@dude.podzone.net>.
On 6/21/2011 3:14 PM, Anand Somani wrote:
> Not sure if it is that simple, a quorum can fail with writes happening
> on some nodes (there is no rollback). Also there is no concept of
> atomic compare-and-swap.
>
Good points. I suppose what I need is for the client to implement the
part of ACID that C* does not. So, right off the bat, can anyone tell
me if that is even possible conceptually? If so, please throw out some
terms that I can wiki and some Java API's would be much appreciated as
well. Also, can I accomplish this or make things easier by imposing
some restrictions, such as only allowing single-user access to the data
for certain operations?
Thanks!
Re: Storing Accounting Data
Posted by AJ <aj...@dude.podzone.net>.
And I was thinking of using JTA for transaction processing. I have no
experience with it but on the surface it looks like it should work.
On 6/21/2011 3:31 PM, AJ wrote:
> What's the best accepted way to handle that 100% in the client? Retries?
>
> On 6/21/2011 3:14 PM, Anand Somani wrote:
>> Not sure if it is that simple, a quorum can fail with writes
>> happening on some nodes (there is no rollback). Also there is no
>> concept of atomic compare-and-swap.
>>
>> On Tue, Jun 21, 2011 at 2:03 PM, AJ <aj@dude.podzone.net
>> <ma...@dude.podzone.net>> wrote:
>>
>> On 6/21/2011 2:50 PM, Stephen Connolly wrote:
>>>
>>> how important are things like transactional consistency for you?
>>>
>>> would you have issues if only one side of a transfer was recorded?
>>>
>>
>> Right. Both of those questions are about consistency. Isn't the
>> simple solution is to use QUORUM read/writes?
>>
>>> cassandra, out of the box, on it's own, would not be ideal if
>>> the above two things are important for you.
>>>
>>> you can add components to a system to help address these things,
>>> eg zookeeper, etc. a reason why you moght do this is if you
>>> already use cassandra in your app and are trying to limit the
>>> number of databases
>>>
>>> - Stephen
>>>
>>> ---
>>> Sent from my Android phone, so random spelling mistakes, random
>>> nonsense words and other nonsense are a direct result of using
>>> swype to type on the screen
>>>
>>> On 21 Jun 2011 18:30, "AJ" <aj@dude.podzone.net
>>> <ma...@dude.podzone.net>> wrote:
>>
>>
>
Re: Storing Accounting Data
Posted by AJ <aj...@dude.podzone.net>.
What's the best accepted way to handle that 100% in the client? Retries?
On 6/21/2011 3:14 PM, Anand Somani wrote:
> Not sure if it is that simple, a quorum can fail with writes happening
> on some nodes (there is no rollback). Also there is no concept of
> atomic compare-and-swap.
>
> On Tue, Jun 21, 2011 at 2:03 PM, AJ <aj@dude.podzone.net
> <ma...@dude.podzone.net>> wrote:
>
> On 6/21/2011 2:50 PM, Stephen Connolly wrote:
>>
>> how important are things like transactional consistency for you?
>>
>> would you have issues if only one side of a transfer was recorded?
>>
>
> Right. Both of those questions are about consistency. Isn't the
> simple solution is to use QUORUM read/writes?
>
>> cassandra, out of the box, on it's own, would not be ideal if the
>> above two things are important for you.
>>
>> you can add components to a system to help address these things,
>> eg zookeeper, etc. a reason why you moght do this is if you
>> already use cassandra in your app and are trying to limit the
>> number of databases
>>
>> - Stephen
>>
>> ---
>> Sent from my Android phone, so random spelling mistakes, random
>> nonsense words and other nonsense are a direct result of using
>> swype to type on the screen
>>
>> On 21 Jun 2011 18:30, "AJ" <aj@dude.podzone.net
>> <ma...@dude.podzone.net>> wrote:
>
>
Re: Storing Accounting Data
Posted by Anand Somani <me...@gmail.com>.
Not sure if it is that simple, a quorum can fail with writes happening on
some nodes (there is no rollback). Also there is no concept of atomic
compare-and-swap.
On Tue, Jun 21, 2011 at 2:03 PM, AJ <aj...@dude.podzone.net> wrote:
> **
> On 6/21/2011 2:50 PM, Stephen Connolly wrote:
>
> how important are things like transactional consistency for you?
>
> would you have issues if only one side of a transfer was recorded?
>
>
> Right. Both of those questions are about consistency. Isn't the simple
> solution is to use QUORUM read/writes?
>
> cassandra, out of the box, on it's own, would not be ideal if the above
> two things are important for you.
>
> you can add components to a system to help address these things, eg
> zookeeper, etc. a reason why you moght do this is if you already use
> cassandra in your app and are trying to limit the number of databases
>
> - Stephen
>
> ---
> Sent from my Android phone, so random spelling mistakes, random nonsense
> words and other nonsense are a direct result of using swype to type on the
> screen
> On 21 Jun 2011 18:30, "AJ" <aj...@dude.podzone.net> wrote:
>
>
>
Re: Storing Accounting Data
Posted by Edward Capriolo <ed...@gmail.com>.
On Tue, Jun 21, 2011 at 10:58 PM, AJ <aj...@dude.podzone.net> wrote:
> **
> On 6/21/2011 3:36 PM, Stephen Connolly wrote:
>
> writes are not atomic.
>
> the first side can succeed at quorum, and the second side can fail
> completely... you'll know it failed, but now what... you retry, still
> failed... erh I'll store it somewhere and retry it later... where do I store
> it?
>
> the consistency level is about tuning whether reads and writes are
> replicated/checked across multiple of the replicates... but at any
> consistency level, each write will either succeed or fail _independently_
>
> you could have one column family which is kind of like a transaction log,
> you write a json object of all the mutations you will make, then you go and
> make the mutations, when they succeed you write a completed column to the
> transaction log... them you can repeat that as often as need
>
> you could have transactions posted as columns in a row, and to get the
> balance you iterate all the columns adding the +'s and -'s
>
> by processing the transaction log, you could establish the highest complete
> timestamp, and add summary balance columns being the running total up to
> that point, so that you don't have to iterate everything
>
> - Stephen
>
>
> Yeah, it's all more than I want to do. But, I just rediscovered Dominic's
> Cages <http://code.google.com/p/cages/>. Has anyone tried it?
>
> ---
> Sent from my Android phone, so random spelling mistakes, random nonsense
> words and other nonsense are a direct result of using swype to type on the
> screen
> On 21 Jun 2011 22:04, "AJ" <aj...@dude.podzone.net> wrote:
>
>
>
I have used cages, it works well. The concept of cages is sound, but if a
large % of your operations have to go through cages and zookeeper locks you
are not going to get the most out of Cassandra. On there other hand if you
need mostly week(er) reads and a small portion of critical actions need to
lock with cages that is a good fit.
Re: Storing Accounting Data
Posted by AJ <aj...@dude.podzone.net>.
On 6/21/2011 3:36 PM, Stephen Connolly wrote:
>
> writes are not atomic.
>
> the first side can succeed at quorum, and the second side can fail
> completely... you'll know it failed, but now what... you retry, still
> failed... erh I'll store it somewhere and retry it later... where do I
> store it?
>
> the consistency level is about tuning whether reads and writes are
> replicated/checked across multiple of the replicates... but at any
> consistency level, each write will either succeed or fail _independently_
>
> you could have one column family which is kind of like a transaction
> log, you write a json object of all the mutations you will make, then
> you go and make the mutations, when they succeed you write a completed
> column to the transaction log... them you can repeat that as often as need
>
> you could have transactions posted as columns in a row, and to get the
> balance you iterate all the columns adding the +'s and -'s
>
> by processing the transaction log, you could establish the highest
> complete timestamp, and add summary balance columns being the running
> total up to that point, so that you don't have to iterate everything
>
> - Stephen
>
Yeah, it's all more than I want to do. But, I just rediscovered
Dominic's Cages <http://code.google.com/p/cages/>. Has anyone tried it?
>
> ---
> Sent from my Android phone, so random spelling mistakes, random
> nonsense words and other nonsense are a direct result of using swype
> to type on the screen
>
> On 21 Jun 2011 22:04, "AJ" <aj@dude.podzone.net
> <ma...@dude.podzone.net>> wrote:
Re: Storing Accounting Data
Posted by Stephen Connolly <st...@gmail.com>.
writes are not atomic.
the first side can succeed at quorum, and the second side can fail
completely... you'll know it failed, but now what... you retry, still
failed... erh I'll store it somewhere and retry it later... where do I store
it?
the consistency level is about tuning whether reads and writes are
replicated/checked across multiple of the replicates... but at any
consistency level, each write will either succeed or fail _independently_
you could have one column family which is kind of like a transaction log,
you write a json object of all the mutations you will make, then you go and
make the mutations, when they succeed you write a completed column to the
transaction log... them you can repeat that as often as need
you could have transactions posted as columns in a row, and to get the
balance you iterate all the columns adding the +'s and -'s
by processing the transaction log, you could establish the highest complete
timestamp, and add summary balance columns being the running total up to
that point, so that you don't have to iterate everything
- Stephen
---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 21 Jun 2011 22:04, "AJ" <aj...@dude.podzone.net> wrote:
Re: Storing Accounting Data
Posted by AJ <aj...@dude.podzone.net>.
On 6/21/2011 2:50 PM, Stephen Connolly wrote:
>
> how important are things like transactional consistency for you?
>
> would you have issues if only one side of a transfer was recorded?
>
Right. Both of those questions are about consistency. Isn't the simple
solution is to use QUORUM read/writes?
> cassandra, out of the box, on it's own, would not be ideal if the
> above two things are important for you.
>
> you can add components to a system to help address these things, eg
> zookeeper, etc. a reason why you moght do this is if you already use
> cassandra in your app and are trying to limit the number of databases
>
> - Stephen
>
> ---
> Sent from my Android phone, so random spelling mistakes, random
> nonsense words and other nonsense are a direct result of using swype
> to type on the screen
>
> On 21 Jun 2011 18:30, "AJ" <aj@dude.podzone.net
> <ma...@dude.podzone.net>> wrote:
Re: Storing Accounting Data
Posted by Stephen Connolly <st...@gmail.com>.
how important are things like transactional consistency for you?
would you have issues if only one side of a transfer was recorded?
cassandra, out of the box, on it's own, would not be ideal if the above two
things are important for you.
you can add components to a system to help address these things, eg
zookeeper, etc. a reason why you moght do this is if you already use
cassandra in your app and are trying to limit the number of databases
- Stephen
---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 21 Jun 2011 18:30, "AJ" <aj...@dude.podzone.net> wrote: