You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by JKnight JKnight <be...@gmail.com> on 2010/04/05 04:48:05 UTC
Cassandra Design or another solution
Dear all,
I want to design the data storage to store user's mark for a large amount of
user. When system run, user's mark changes frequently.
I want to list top 10 user have largest mark.
Could we use Cassandra for store this data?
Ex, here my Cassandra data model design:
Mark{
userId{
mark_userId
},
}
When user's mark changes, we remove old mark_userId and add new mark_userId.
Because user's mark change frequently and with large amount of user, I
think Cassandra can not satisfy.
Could anybody show me another solution to do that?
Thank a lot for support.
--
Best regards,
JKnight
Re: Cassandra Design or another solution
Posted by JKnight JKnight <be...@gmail.com>.
Thanks for your help, David.
On Mon, Apr 5, 2010 at 7:25 PM, Andriy Bohdan <a....@gmail.com> wrote:
> It makes sense.
>
> Thanks, David!
>
> On Mon, Apr 5, 2010 at 2:34 PM, David Timothy Strauss
> <da...@fourkitchens.com> wrote:
> > Cache the <product, tag> => <score> map as you write values (a
> "write-through" cache) so that reading the current score hits something like
> memcached instead of Cassandra. With a cache hit, you get an ideal,
> write-only path in Cassandra. Three blind writes in Cassandra is cheap -- no
> matter what your scale. The only risk is inability to efficiently remove old
> scores if you lose the contents of memcached, but that risk can be mitigated
> various ways.
> >
> > Of course, I'm assuming a single data center, here. Memcached isn't too
> useful for this if you need to update scores at two data centers.
> >
> > I'm not sure how much the 0.6 row cache might help in this case, too.
> >
> > ----- "Andriy Bohdan" <a....@gmail.com> wrote:
> >
> >> Hello guys
> >>
> >> I have a pretty similar task. There's a need to store tags of
> >> products
> >> with score. Score may go up and down and tags have to be ordered by
> >> their score for each product. Score is updated "very" often.
> >>
> >> I was thinking of using the following model (simplified here for
> >> clarity):
> >>
> >> Product = {
> >> product_key: {
> >> name: ....
> >> etc..
> >> }
> >> ...
> >> }
> >>
> >> Product_Tags = {
> >> product_key : {
> >> tag_name: score
> >> ...
> >> }
> >> ...
> >> }
> >>
> >> Product_Tags_Ordered (compareWith:BytesType) = {
> >> product_key: {
> >> (score, time_uuid) : tag_name
> >> ...
> >> }
> >> ...
> >> }
> >>
> >> So to update a score of a tag:
> >> 1) need to look old value of score to be able to remove it from
> >> Product_Tags_Ordered
> >> 2) remove Row from Product_Tags_Ordered with old score
> >> 3) update score in Product_Tags
> >> 4) insert new Row into Product_Tags_Ordered with new score
> >>
> >> 4 IO operations look like a bit too much to update one score as for
> >> me.
> >>
> >> I'm curious if there's any better solution I missed.
> >>
> >>
> >> On Mon, Apr 5, 2010 at 11:54 AM, JKnight JKnight <be...@gmail.com>
> >> wrote:
> >> > Thanks David,
> >> > But what's does "monotonicity" mean?
> >> >
> >> > User's score belongs to their action. When they win the game or
> >> sale
> >> > something, user's score will increase. When user lose the game or
> >> buy
> >> > something, user's score will decrease.
> >> >
> >> > On Mon, Apr 5, 2010 at 4:09 AM, David Strauss
> >> <da...@fourkitchens.com>
> >> > wrote:
> >> >>
> >> >> I need the question about monotonicity answered, too.
> >> >>
> >> >> You should also know: Cassandra is not ideal for directly tracking
> >> >> values you increment or decrement.
> >> >>
> >> >> On 2010-04-05 08:04, JKnight JKnight wrote:
> >> >> > Thanks for for reply, David.
> >> >> >
> >> >> > I will tell more the detail about the system. My system is used
> >> to store
> >> >> > the score (point) user earn when they play game.
> >> >> >
> >> >> > "Mark" is the score.
> >> >> > User's score changes when user win game, buy or sell anything.
> >> >> >
> >> >> > Sorry I make a mistake. My data model is:
> >> >> >
> >> >> > Mark{ //Column Family
> >> >> > gameId:{ //row key
> >> >> > mark_userId: ""// (column name : value),
> >> >> > mark2_userId2: ""
> >> >> > },
> >> >> > gameId2:{//row key
> >> >> > mark_userId: ""
> >> >> > }
> >> >> > }
> >> >> >
> >> >> >
> >> >> > On Sun, Apr 4, 2010 at 11:44 PM, David Strauss
> >> <david@fourkitchens.com
> >> >> > <ma...@fourkitchens.com>> wrote:
> >> >> >
> >> >> > On 2010-04-05 02:48, JKnight JKnight wrote:
> >> >> > > I want to design the data storage to store user's mark for
> >> a large
> >> >> > > amount of user. When system run, user's mark changes
> >> frequently.
> >> >> >
> >> >> > What is a "mark"?
> >> >> >
> >> >> > > I want to list top 10 user have largest mark.
> >> >> >
> >> >> > Do the "marks" increase monotonically? What other properties
> >> do they
> >> >> > have?
> >> >> >
> >> >> > > Could we use Cassandra for store this data?
> >> >> > >
> >> >> > > Ex, here my Cassandra data model design:
> >> >> > > Mark{
> >> >> > > userId{
> >> >> > > mark_userId
> >> >> > > },
> >> >> > > }
> >> >> >
> >> >> > I do not understand that notation. What parts are the CF,
> >> key/row,
> >> >> > and
> >> >> > column?
> >> >> >
> >> >> > > When user's mark changes, we remove old mark_userId and add
> >> new
> >> >> > > mark_userId.
> >> >> > > Because user's mark change frequently and with large amount
> >> of
> >> >> > user, I
> >> >> > > think Cassandra can not satisfy.
> >> >> >
> >> >> > On the contrary, Cassandra excels at tracking rapidly
> >> changing data
> >> >> > and
> >> >> > even shards rows to scale I/O horizontally.
> >> >> >
> >> >> > --
> >> >> > David Strauss
> >> >> > | david@fourkitchens.com <ma...@fourkitchens.com>
> >> >> > Four Kitchens
> >> >> > | http://fourkitchens.com
> >> >> > | +1 512 454 6659 [office]
> >> >> > | +1 512 870 8453 [direct]
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Best regards,
> >> >> > JKnight
> >> >>
> >> >>
> >> >> --
> >> >> David Strauss
> >> >> | david@fourkitchens.com
> >> >> | +1 512 577 5827 [mobile]
> >> >> Four Kitchens
> >> >> | http://fourkitchens.com
> >> >> | +1 512 454 6659 [office]
> >> >> | +1 512 870 8453 [direct]
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Best regards,
> >> > JKnight
> >> >
> >>
> >>
> >>
> >> --
> >> Andriy
> >
> > --
> > David Strauss
> > | david@fourkitchens.com
> > | +1 512 577 5827 [mobile]
> > Four Kitchens
> > | http://fourkitchens.com
> > | +1 512 454 6659 [office]
> > | +1 512 870 8453 [direct]
> >
>
>
>
> --
> Andriy
>
--
Best regards,
JKnight
Re: Cassandra Design or another solution
Posted by Andriy Bohdan <a....@gmail.com>.
It makes sense.
Thanks, David!
On Mon, Apr 5, 2010 at 2:34 PM, David Timothy Strauss
<da...@fourkitchens.com> wrote:
> Cache the <product, tag> => <score> map as you write values (a "write-through" cache) so that reading the current score hits something like memcached instead of Cassandra. With a cache hit, you get an ideal, write-only path in Cassandra. Three blind writes in Cassandra is cheap -- no matter what your scale. The only risk is inability to efficiently remove old scores if you lose the contents of memcached, but that risk can be mitigated various ways.
>
> Of course, I'm assuming a single data center, here. Memcached isn't too useful for this if you need to update scores at two data centers.
>
> I'm not sure how much the 0.6 row cache might help in this case, too.
>
> ----- "Andriy Bohdan" <a....@gmail.com> wrote:
>
>> Hello guys
>>
>> I have a pretty similar task. There's a need to store tags of
>> products
>> with score. Score may go up and down and tags have to be ordered by
>> their score for each product. Score is updated "very" often.
>>
>> I was thinking of using the following model (simplified here for
>> clarity):
>>
>> Product = {
>> product_key: {
>> name: ....
>> etc..
>> }
>> ...
>> }
>>
>> Product_Tags = {
>> product_key : {
>> tag_name: score
>> ...
>> }
>> ...
>> }
>>
>> Product_Tags_Ordered (compareWith:BytesType) = {
>> product_key: {
>> (score, time_uuid) : tag_name
>> ...
>> }
>> ...
>> }
>>
>> So to update a score of a tag:
>> 1) need to look old value of score to be able to remove it from
>> Product_Tags_Ordered
>> 2) remove Row from Product_Tags_Ordered with old score
>> 3) update score in Product_Tags
>> 4) insert new Row into Product_Tags_Ordered with new score
>>
>> 4 IO operations look like a bit too much to update one score as for
>> me.
>>
>> I'm curious if there's any better solution I missed.
>>
>>
>> On Mon, Apr 5, 2010 at 11:54 AM, JKnight JKnight <be...@gmail.com>
>> wrote:
>> > Thanks David,
>> > But what's does "monotonicity" mean?
>> >
>> > User's score belongs to their action. When they win the game or
>> sale
>> > something, user's score will increase. When user lose the game or
>> buy
>> > something, user's score will decrease.
>> >
>> > On Mon, Apr 5, 2010 at 4:09 AM, David Strauss
>> <da...@fourkitchens.com>
>> > wrote:
>> >>
>> >> I need the question about monotonicity answered, too.
>> >>
>> >> You should also know: Cassandra is not ideal for directly tracking
>> >> values you increment or decrement.
>> >>
>> >> On 2010-04-05 08:04, JKnight JKnight wrote:
>> >> > Thanks for for reply, David.
>> >> >
>> >> > I will tell more the detail about the system. My system is used
>> to store
>> >> > the score (point) user earn when they play game.
>> >> >
>> >> > "Mark" is the score.
>> >> > User's score changes when user win game, buy or sell anything.
>> >> >
>> >> > Sorry I make a mistake. My data model is:
>> >> >
>> >> > Mark{ //Column Family
>> >> > gameId:{ //row key
>> >> > mark_userId: ""// (column name : value),
>> >> > mark2_userId2: ""
>> >> > },
>> >> > gameId2:{//row key
>> >> > mark_userId: ""
>> >> > }
>> >> > }
>> >> >
>> >> >
>> >> > On Sun, Apr 4, 2010 at 11:44 PM, David Strauss
>> <david@fourkitchens.com
>> >> > <ma...@fourkitchens.com>> wrote:
>> >> >
>> >> > On 2010-04-05 02:48, JKnight JKnight wrote:
>> >> > > I want to design the data storage to store user's mark for
>> a large
>> >> > > amount of user. When system run, user's mark changes
>> frequently.
>> >> >
>> >> > What is a "mark"?
>> >> >
>> >> > > I want to list top 10 user have largest mark.
>> >> >
>> >> > Do the "marks" increase monotonically? What other properties
>> do they
>> >> > have?
>> >> >
>> >> > > Could we use Cassandra for store this data?
>> >> > >
>> >> > > Ex, here my Cassandra data model design:
>> >> > > Mark{
>> >> > > userId{
>> >> > > mark_userId
>> >> > > },
>> >> > > }
>> >> >
>> >> > I do not understand that notation. What parts are the CF,
>> key/row,
>> >> > and
>> >> > column?
>> >> >
>> >> > > When user's mark changes, we remove old mark_userId and add
>> new
>> >> > > mark_userId.
>> >> > > Because user's mark change frequently and with large amount
>> of
>> >> > user, I
>> >> > > think Cassandra can not satisfy.
>> >> >
>> >> > On the contrary, Cassandra excels at tracking rapidly
>> changing data
>> >> > and
>> >> > even shards rows to scale I/O horizontally.
>> >> >
>> >> > --
>> >> > David Strauss
>> >> > | david@fourkitchens.com <ma...@fourkitchens.com>
>> >> > Four Kitchens
>> >> > | http://fourkitchens.com
>> >> > | +1 512 454 6659 [office]
>> >> > | +1 512 870 8453 [direct]
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best regards,
>> >> > JKnight
>> >>
>> >>
>> >> --
>> >> David Strauss
>> >> | david@fourkitchens.com
>> >> | +1 512 577 5827 [mobile]
>> >> Four Kitchens
>> >> | http://fourkitchens.com
>> >> | +1 512 454 6659 [office]
>> >> | +1 512 870 8453 [direct]
>> >>
>> >
>> >
>> >
>> > --
>> > Best regards,
>> > JKnight
>> >
>>
>>
>>
>> --
>> Andriy
>
> --
> David Strauss
> | david@fourkitchens.com
> | +1 512 577 5827 [mobile]
> Four Kitchens
> | http://fourkitchens.com
> | +1 512 454 6659 [office]
> | +1 512 870 8453 [direct]
>
--
Andriy
Re: Cassandra Design or another solution
Posted by David Timothy Strauss <da...@fourkitchens.com>.
Cache the <product, tag> => <score> map as you write values (a "write-through" cache) so that reading the current score hits something like memcached instead of Cassandra. With a cache hit, you get an ideal, write-only path in Cassandra. Three blind writes in Cassandra is cheap -- no matter what your scale. The only risk is inability to efficiently remove old scores if you lose the contents of memcached, but that risk can be mitigated various ways.
Of course, I'm assuming a single data center, here. Memcached isn't too useful for this if you need to update scores at two data centers.
I'm not sure how much the 0.6 row cache might help in this case, too.
----- "Andriy Bohdan" <a....@gmail.com> wrote:
> Hello guys
>
> I have a pretty similar task. There's a need to store tags of
> products
> with score. Score may go up and down and tags have to be ordered by
> their score for each product. Score is updated "very" often.
>
> I was thinking of using the following model (simplified here for
> clarity):
>
> Product = {
> product_key: {
> name: ....
> etc..
> }
> ...
> }
>
> Product_Tags = {
> product_key : {
> tag_name: score
> ...
> }
> ...
> }
>
> Product_Tags_Ordered (compareWith:BytesType) = {
> product_key: {
> (score, time_uuid) : tag_name
> ...
> }
> ...
> }
>
> So to update a score of a tag:
> 1) need to look old value of score to be able to remove it from
> Product_Tags_Ordered
> 2) remove Row from Product_Tags_Ordered with old score
> 3) update score in Product_Tags
> 4) insert new Row into Product_Tags_Ordered with new score
>
> 4 IO operations look like a bit too much to update one score as for
> me.
>
> I'm curious if there's any better solution I missed.
>
>
> On Mon, Apr 5, 2010 at 11:54 AM, JKnight JKnight <be...@gmail.com>
> wrote:
> > Thanks David,
> > But what's does "monotonicity" mean?
> >
> > User's score belongs to their action. When they win the game or
> sale
> > something, user's score will increase. When user lose the game or
> buy
> > something, user's score will decrease.
> >
> > On Mon, Apr 5, 2010 at 4:09 AM, David Strauss
> <da...@fourkitchens.com>
> > wrote:
> >>
> >> I need the question about monotonicity answered, too.
> >>
> >> You should also know: Cassandra is not ideal for directly tracking
> >> values you increment or decrement.
> >>
> >> On 2010-04-05 08:04, JKnight JKnight wrote:
> >> > Thanks for for reply, David.
> >> >
> >> > I will tell more the detail about the system. My system is used
> to store
> >> > the score (point) user earn when they play game.
> >> >
> >> > "Mark" is the score.
> >> > User's score changes when user win game, buy or sell anything.
> >> >
> >> > Sorry I make a mistake. My data model is:
> >> >
> >> > Mark{ //Column Family
> >> > gameId:{ //row key
> >> > mark_userId: ""// (column name : value),
> >> > mark2_userId2: ""
> >> > },
> >> > gameId2:{//row key
> >> > mark_userId: ""
> >> > }
> >> > }
> >> >
> >> >
> >> > On Sun, Apr 4, 2010 at 11:44 PM, David Strauss
> <david@fourkitchens.com
> >> > <ma...@fourkitchens.com>> wrote:
> >> >
> >> > On 2010-04-05 02:48, JKnight JKnight wrote:
> >> > > I want to design the data storage to store user's mark for
> a large
> >> > > amount of user. When system run, user's mark changes
> frequently.
> >> >
> >> > What is a "mark"?
> >> >
> >> > > I want to list top 10 user have largest mark.
> >> >
> >> > Do the "marks" increase monotonically? What other properties
> do they
> >> > have?
> >> >
> >> > > Could we use Cassandra for store this data?
> >> > >
> >> > > Ex, here my Cassandra data model design:
> >> > > Mark{
> >> > > userId{
> >> > > mark_userId
> >> > > },
> >> > > }
> >> >
> >> > I do not understand that notation. What parts are the CF,
> key/row,
> >> > and
> >> > column?
> >> >
> >> > > When user's mark changes, we remove old mark_userId and add
> new
> >> > > mark_userId.
> >> > > Because user's mark change frequently and with large amount
> of
> >> > user, I
> >> > > think Cassandra can not satisfy.
> >> >
> >> > On the contrary, Cassandra excels at tracking rapidly
> changing data
> >> > and
> >> > even shards rows to scale I/O horizontally.
> >> >
> >> > --
> >> > David Strauss
> >> > | david@fourkitchens.com <ma...@fourkitchens.com>
> >> > Four Kitchens
> >> > | http://fourkitchens.com
> >> > | +1 512 454 6659 [office]
> >> > | +1 512 870 8453 [direct]
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Best regards,
> >> > JKnight
> >>
> >>
> >> --
> >> David Strauss
> >> | david@fourkitchens.com
> >> | +1 512 577 5827 [mobile]
> >> Four Kitchens
> >> | http://fourkitchens.com
> >> | +1 512 454 6659 [office]
> >> | +1 512 870 8453 [direct]
> >>
> >
> >
> >
> > --
> > Best regards,
> > JKnight
> >
>
>
>
> --
> Andriy
--
David Strauss
| david@fourkitchens.com
| +1 512 577 5827 [mobile]
Four Kitchens
| http://fourkitchens.com
| +1 512 454 6659 [office]
| +1 512 870 8453 [direct]
Re: Cassandra Design or another solution
Posted by Andriy Bohdan <a....@gmail.com>.
Hello guys
I have a pretty similar task. There's a need to store tags of products
with score. Score may go up and down and tags have to be ordered by
their score for each product. Score is updated "very" often.
I was thinking of using the following model (simplified here for clarity):
Product = {
product_key: {
name: ....
etc..
}
...
}
Product_Tags = {
product_key : {
tag_name: score
...
}
...
}
Product_Tags_Ordered (compareWith:BytesType) = {
product_key: {
(score, time_uuid) : tag_name
...
}
...
}
So to update a score of a tag:
1) need to look old value of score to be able to remove it from
Product_Tags_Ordered
2) remove Row from Product_Tags_Ordered with old score
3) update score in Product_Tags
4) insert new Row into Product_Tags_Ordered with new score
4 IO operations look like a bit too much to update one score as for me.
I'm curious if there's any better solution I missed.
On Mon, Apr 5, 2010 at 11:54 AM, JKnight JKnight <be...@gmail.com> wrote:
> Thanks David,
> But what's does "monotonicity" mean?
>
> User's score belongs to their action. When they win the game or sale
> something, user's score will increase. When user lose the game or buy
> something, user's score will decrease.
>
> On Mon, Apr 5, 2010 at 4:09 AM, David Strauss <da...@fourkitchens.com>
> wrote:
>>
>> I need the question about monotonicity answered, too.
>>
>> You should also know: Cassandra is not ideal for directly tracking
>> values you increment or decrement.
>>
>> On 2010-04-05 08:04, JKnight JKnight wrote:
>> > Thanks for for reply, David.
>> >
>> > I will tell more the detail about the system. My system is used to store
>> > the score (point) user earn when they play game.
>> >
>> > "Mark" is the score.
>> > User's score changes when user win game, buy or sell anything.
>> >
>> > Sorry I make a mistake. My data model is:
>> >
>> > Mark{ //Column Family
>> > gameId:{ //row key
>> > mark_userId: ""// (column name : value),
>> > mark2_userId2: ""
>> > },
>> > gameId2:{//row key
>> > mark_userId: ""
>> > }
>> > }
>> >
>> >
>> > On Sun, Apr 4, 2010 at 11:44 PM, David Strauss <david@fourkitchens.com
>> > <ma...@fourkitchens.com>> wrote:
>> >
>> > On 2010-04-05 02:48, JKnight JKnight wrote:
>> > > I want to design the data storage to store user's mark for a large
>> > > amount of user. When system run, user's mark changes frequently.
>> >
>> > What is a "mark"?
>> >
>> > > I want to list top 10 user have largest mark.
>> >
>> > Do the "marks" increase monotonically? What other properties do they
>> > have?
>> >
>> > > Could we use Cassandra for store this data?
>> > >
>> > > Ex, here my Cassandra data model design:
>> > > Mark{
>> > > userId{
>> > > mark_userId
>> > > },
>> > > }
>> >
>> > I do not understand that notation. What parts are the CF, key/row,
>> > and
>> > column?
>> >
>> > > When user's mark changes, we remove old mark_userId and add new
>> > > mark_userId.
>> > > Because user's mark change frequently and with large amount of
>> > user, I
>> > > think Cassandra can not satisfy.
>> >
>> > On the contrary, Cassandra excels at tracking rapidly changing data
>> > and
>> > even shards rows to scale I/O horizontally.
>> >
>> > --
>> > David Strauss
>> > | david@fourkitchens.com <ma...@fourkitchens.com>
>> > Four Kitchens
>> > | http://fourkitchens.com
>> > | +1 512 454 6659 [office]
>> > | +1 512 870 8453 [direct]
>> >
>> >
>> >
>> >
>> > --
>> > Best regards,
>> > JKnight
>>
>>
>> --
>> David Strauss
>> | david@fourkitchens.com
>> | +1 512 577 5827 [mobile]
>> Four Kitchens
>> | http://fourkitchens.com
>> | +1 512 454 6659 [office]
>> | +1 512 870 8453 [direct]
>>
>
>
>
> --
> Best regards,
> JKnight
>
--
Andriy
Re: Cassandra Design or another solution
Posted by David Timothy Strauss <da...@fourkitchens.com>.
In any case, the common approach to this in Cassandra is to not directly manipulate the user's total score but to insert columns representing changes to the score, later totaling them (and possibly inserting them elsewhere so you get the automatic sort). There are many fancy ways to approach this problem and reduce recalculation work.
To be more specific about my reason for asking about monotonicity, if user scores increased monotonically, you could simply keep a current "top ten" list and bump out old members as new people's scores qualified for the top ten. Thus, you'd be keeping an ordered set of the top ten instead of an ordered set of all people. Unfortunately, the possibility of decreasing scores means members of the top ten may self-disqualify by a score decline, requiring promotion of the former 11th person to the top ten and immediate identification of the former 12th person to fill the 11th spot. (With a lack of monotonicity, the top eleven -- not just ten -- must always be tracked to efficiently know when someone drops below the top ten.)
----- "David Timothy Strauss" <da...@fourkitchens.com> wrote:
If user scores move in more than one direction, as they apparently do in your case, they are not monotonic. Monotonicity can make system design a bit easier for various reasons.
----- "JKnight JKnight" <be...@gmail.com> wrote:
Thanks David,
But what's does "monotonicity" mean?
User's score belongs to their action. When they win the game or sale something, user's score will increase. When user lose the game or buy something, user's score will decrease.
On Mon, Apr 5, 2010 at 4:09 AM, David Strauss < david@fourkitchens.com > wrote:
I need the question about monotonicity answered, too.
You should also know: Cassandra is not ideal for directly tracking
values you increment or decrement.
On 2010-04-05 08:04, JKnight JKnight wrote:
> Thanks for for reply, David.
>
> I will tell more the detail about the system. My system is used to store
> the score (point) user earn when they play game.
>
> "Mark" is the score.
> User's score changes when user win game, buy or sell anything.
>
> Sorry I make a mistake. My data model is:
>
> Mark{ //Column Family
> gameId:{ //row key
> mark_userId: ""// (column name : value),
> mark2_userId2: ""
> },
> gameId2:{//row key
> mark_userId: ""
> }
> }
>
>
> On Sun, Apr 4, 2010 at 11:44 PM, David Strauss < david@fourkitchens.com
> <mailto: david@fourkitchens.com >> wrote:
>
> On 2010-04-05 02:48, JKnight JKnight wrote:
> > I want to design the data storage to store user's mark for a large
> > amount of user. When system run, user's mark changes frequently.
>
> What is a "mark"?
>
> > I want to list top 10 user have largest mark.
>
> Do the "marks" increase monotonically? What other properties do they
> have?
>
> > Could we use Cassandra for store this data?
> >
> > Ex, here my Cassandra data model design:
> > Mark{
> > userId{
> > mark_userId
> > },
> > }
>
> I do not understand that notation. What parts are the CF, key/row, and
> column?
>
> > When user's mark changes, we remove old mark_userId and add new
> > mark_userId.
> > Because user's mark change frequently and with large amount of
> user, I
> > think Cassandra can not satisfy.
>
> On the contrary, Cassandra excels at tracking rapidly changing data and
> even shards rows to scale I/O horizontally.
>
> --
> David Strauss
> | david@fourkitchens.com <mailto: david@fourkitchens.com >
> Four Kitchens
> | http://fourkitchens.com
> | +1 512 454 6659 [office]
> | +1 512 870 8453 [direct]
>
>
>
>
> --
> Best regards,
> JKnight
--
David Strauss
| david@fourkitchens.com
| +1 512 577 5827 [mobile]
Four Kitchens
| http://fourkitchens.com
| +1 512 454 6659 [office]
| +1 512 870 8453 [direct]
--
Best regards,
JKnight
--
David Strauss
| david@fourkitchens.com
| +1 512 577 5827 [mobile]
Four Kitchens
| http://fourkitchens.com
| +1 512 454 6659 [office]
| +1 512 870 8453 [direct]
--
David Strauss
| david@fourkitchens.com
| +1 512 577 5827 [mobile]
Four Kitchens
| http://fourkitchens.com
| +1 512 454 6659 [office]
| +1 512 870 8453 [direct]
Re: Cassandra Design or another solution
Posted by David Timothy Strauss <da...@fourkitchens.com>.
If user scores move in more than one direction, as they apparently do in your case, they are not monotonic. Monotonicity can make system design a bit easier for various reasons.
----- "JKnight JKnight" <be...@gmail.com> wrote:
Thanks David,
But what's does "monotonicity" mean?
User's score belongs to their action. When they win the game or sale something, user's score will increase. When user lose the game or buy something, user's score will decrease.
On Mon, Apr 5, 2010 at 4:09 AM, David Strauss < david@fourkitchens.com > wrote:
I need the question about monotonicity answered, too.
You should also know: Cassandra is not ideal for directly tracking
values you increment or decrement.
On 2010-04-05 08:04, JKnight JKnight wrote:
> Thanks for for reply, David.
>
> I will tell more the detail about the system. My system is used to store
> the score (point) user earn when they play game.
>
> "Mark" is the score.
> User's score changes when user win game, buy or sell anything.
>
> Sorry I make a mistake. My data model is:
>
> Mark{ //Column Family
> gameId:{ //row key
> mark_userId: ""// (column name : value),
> mark2_userId2: ""
> },
> gameId2:{//row key
> mark_userId: ""
> }
> }
>
>
> On Sun, Apr 4, 2010 at 11:44 PM, David Strauss < david@fourkitchens.com
> <mailto: david@fourkitchens.com >> wrote:
>
> On 2010-04-05 02:48, JKnight JKnight wrote:
> > I want to design the data storage to store user's mark for a large
> > amount of user. When system run, user's mark changes frequently.
>
> What is a "mark"?
>
> > I want to list top 10 user have largest mark.
>
> Do the "marks" increase monotonically? What other properties do they
> have?
>
> > Could we use Cassandra for store this data?
> >
> > Ex, here my Cassandra data model design:
> > Mark{
> > userId{
> > mark_userId
> > },
> > }
>
> I do not understand that notation. What parts are the CF, key/row, and
> column?
>
> > When user's mark changes, we remove old mark_userId and add new
> > mark_userId.
> > Because user's mark change frequently and with large amount of
> user, I
> > think Cassandra can not satisfy.
>
> On the contrary, Cassandra excels at tracking rapidly changing data and
> even shards rows to scale I/O horizontally.
>
> --
> David Strauss
> | david@fourkitchens.com <mailto: david@fourkitchens.com >
> Four Kitchens
> | http://fourkitchens.com
> | +1 512 454 6659 [office]
> | +1 512 870 8453 [direct]
>
>
>
>
> --
> Best regards,
> JKnight
--
David Strauss
| david@fourkitchens.com
| +1 512 577 5827 [mobile]
Four Kitchens
| http://fourkitchens.com
| +1 512 454 6659 [office]
| +1 512 870 8453 [direct]
--
Best regards,
JKnight
--
David Strauss
| david@fourkitchens.com
| +1 512 577 5827 [mobile]
Four Kitchens
| http://fourkitchens.com
| +1 512 454 6659 [office]
| +1 512 870 8453 [direct]
Re: Cassandra Design or another solution
Posted by JKnight JKnight <be...@gmail.com>.
Thanks David,
But what's does "monotonicity" mean?
User's score belongs to their action. When they win the game or sale
something, user's score will increase. When user lose the game or buy
something, user's score will decrease.
On Mon, Apr 5, 2010 at 4:09 AM, David Strauss <da...@fourkitchens.com>wrote:
> I need the question about monotonicity answered, too.
>
> You should also know: Cassandra is not ideal for directly tracking
> values you increment or decrement.
>
> On 2010-04-05 08:04, JKnight JKnight wrote:
> > Thanks for for reply, David.
> >
> > I will tell more the detail about the system. My system is used to store
> > the score (point) user earn when they play game.
> >
> > "Mark" is the score.
> > User's score changes when user win game, buy or sell anything.
> >
> > Sorry I make a mistake. My data model is:
> >
> > Mark{ //Column Family
> > gameId:{ //row key
> > mark_userId: ""// (column name : value),
> > mark2_userId2: ""
> > },
> > gameId2:{//row key
> > mark_userId: ""
> > }
> > }
> >
> >
> > On Sun, Apr 4, 2010 at 11:44 PM, David Strauss <david@fourkitchens.com
> > <ma...@fourkitchens.com>> wrote:
> >
> > On 2010-04-05 02:48, JKnight JKnight wrote:
> > > I want to design the data storage to store user's mark for a large
> > > amount of user. When system run, user's mark changes frequently.
> >
> > What is a "mark"?
> >
> > > I want to list top 10 user have largest mark.
> >
> > Do the "marks" increase monotonically? What other properties do they
> > have?
> >
> > > Could we use Cassandra for store this data?
> > >
> > > Ex, here my Cassandra data model design:
> > > Mark{
> > > userId{
> > > mark_userId
> > > },
> > > }
> >
> > I do not understand that notation. What parts are the CF, key/row,
> and
> > column?
> >
> > > When user's mark changes, we remove old mark_userId and add new
> > > mark_userId.
> > > Because user's mark change frequently and with large amount of
> > user, I
> > > think Cassandra can not satisfy.
> >
> > On the contrary, Cassandra excels at tracking rapidly changing data
> and
> > even shards rows to scale I/O horizontally.
> >
> > --
> > David Strauss
> > | david@fourkitchens.com <ma...@fourkitchens.com>
> > Four Kitchens
> > | http://fourkitchens.com
> > | +1 512 454 6659 [office]
> > | +1 512 870 8453 [direct]
> >
> >
> >
> >
> > --
> > Best regards,
> > JKnight
>
>
> --
> David Strauss
> | david@fourkitchens.com
> | +1 512 577 5827 [mobile]
> Four Kitchens
> | http://fourkitchens.com
> | +1 512 454 6659 [office]
> | +1 512 870 8453 [direct]
>
>
--
Best regards,
JKnight
Re: Cassandra Design or another solution
Posted by David Strauss <da...@fourkitchens.com>.
I need the question about monotonicity answered, too.
You should also know: Cassandra is not ideal for directly tracking
values you increment or decrement.
On 2010-04-05 08:04, JKnight JKnight wrote:
> Thanks for for reply, David.
>
> I will tell more the detail about the system. My system is used to store
> the score (point) user earn when they play game.
>
> "Mark" is the score.
> User's score changes when user win game, buy or sell anything.
>
> Sorry I make a mistake. My data model is:
>
> Mark{ //Column Family
> gameId:{ //row key
> mark_userId: ""// (column name : value),
> mark2_userId2: ""
> },
> gameId2:{//row key
> mark_userId: ""
> }
> }
>
>
> On Sun, Apr 4, 2010 at 11:44 PM, David Strauss <david@fourkitchens.com
> <ma...@fourkitchens.com>> wrote:
>
> On 2010-04-05 02:48, JKnight JKnight wrote:
> > I want to design the data storage to store user's mark for a large
> > amount of user. When system run, user's mark changes frequently.
>
> What is a "mark"?
>
> > I want to list top 10 user have largest mark.
>
> Do the "marks" increase monotonically? What other properties do they
> have?
>
> > Could we use Cassandra for store this data?
> >
> > Ex, here my Cassandra data model design:
> > Mark{
> > userId{
> > mark_userId
> > },
> > }
>
> I do not understand that notation. What parts are the CF, key/row, and
> column?
>
> > When user's mark changes, we remove old mark_userId and add new
> > mark_userId.
> > Because user's mark change frequently and with large amount of
> user, I
> > think Cassandra can not satisfy.
>
> On the contrary, Cassandra excels at tracking rapidly changing data and
> even shards rows to scale I/O horizontally.
>
> --
> David Strauss
> | david@fourkitchens.com <ma...@fourkitchens.com>
> Four Kitchens
> | http://fourkitchens.com
> | +1 512 454 6659 [office]
> | +1 512 870 8453 [direct]
>
>
>
>
> --
> Best regards,
> JKnight
--
David Strauss
| david@fourkitchens.com
| +1 512 577 5827 [mobile]
Four Kitchens
| http://fourkitchens.com
| +1 512 454 6659 [office]
| +1 512 870 8453 [direct]
Re: Cassandra Design or another solution
Posted by JKnight JKnight <be...@gmail.com>.
Thanks for for reply, David.
I will tell more the detail about the system. My system is used to store the
score (point) user earn when they play game.
"Mark" is the score.
User's score changes when user win game, buy or sell anything.
Sorry I make a mistake. My data model is:
Mark{ //Column Family
gameId:{ //row key
mark_userId: ""// (column name : value),
mark2_userId2: ""
},
gameId2:{//row key
mark_userId: ""
}
}
On Sun, Apr 4, 2010 at 11:44 PM, David Strauss <da...@fourkitchens.com>wrote:
> On 2010-04-05 02:48, JKnight JKnight wrote:
> > I want to design the data storage to store user's mark for a large
> > amount of user. When system run, user's mark changes frequently.
>
> What is a "mark"?
>
> > I want to list top 10 user have largest mark.
>
> Do the "marks" increase monotonically? What other properties do they have?
>
> > Could we use Cassandra for store this data?
> >
> > Ex, here my Cassandra data model design:
> > Mark{
> > userId{
> > mark_userId
> > },
> > }
>
> I do not understand that notation. What parts are the CF, key/row, and
> column?
>
> > When user's mark changes, we remove old mark_userId and add new
> > mark_userId.
> > Because user's mark change frequently and with large amount of user, I
> > think Cassandra can not satisfy.
>
> On the contrary, Cassandra excels at tracking rapidly changing data and
> even shards rows to scale I/O horizontally.
>
> --
> David Strauss
> | david@fourkitchens.com
> Four Kitchens
> | http://fourkitchens.com
> | +1 512 454 6659 [office]
> | +1 512 870 8453 [direct]
>
>
--
Best regards,
JKnight
Re: Cassandra Design or another solution
Posted by David Strauss <da...@fourkitchens.com>.
On 2010-04-05 02:48, JKnight JKnight wrote:
> I want to design the data storage to store user's mark for a large
> amount of user. When system run, user's mark changes frequently.
What is a "mark"?
> I want to list top 10 user have largest mark.
Do the "marks" increase monotonically? What other properties do they have?
> Could we use Cassandra for store this data?
>
> Ex, here my Cassandra data model design:
> Mark{
> userId{
> mark_userId
> },
> }
I do not understand that notation. What parts are the CF, key/row, and
column?
> When user's mark changes, we remove old mark_userId and add new
> mark_userId.
> Because user's mark change frequently and with large amount of user, I
> think Cassandra can not satisfy.
On the contrary, Cassandra excels at tracking rapidly changing data and
even shards rows to scale I/O horizontally.
--
David Strauss
| david@fourkitchens.com
Four Kitchens
| http://fourkitchens.com
| +1 512 454 6659 [office]
| +1 512 870 8453 [direct]