Using mySQL to emulate Cassandra

[David Boxenhorn <da...@lookin2.com>]

As our launch date approaches, I am getting increasingly nervous about
Cassandra tuning. It is a mysterious black art that I haven't mastered even
at the low usages that we have now. I know of a few more things I can do to
improve things, but how will I know if it is enough? All this is
particularly ironic since - as we are just starting out - we don't have
scalability problems yet, though we hope to!

Luckily, I have completely wrapped Cassandra in an entity mapper, so that I
can easily trade in something else, perhaps temporarily, until we really
need Cassandra's scalability.

So, I'm thinking of emulating Cassandra with mySQL. I would use mySQL either
as a simple key-value store, without joins, or map Cassandra supercolumns to
mySQL columns, probably of type CLOB.

Does anyone want to talk me out of this?

Re: Using mySQL to emulate Cassandra

[Edward Capriolo <ed...@gmail.com>]

On Sun, Nov 28, 2010 at 11:35 AM, Tom Melendez <to...@supertom.com> wrote:
> On Sun, Nov 28, 2010 at 12:28 AM, David Boxenhorn <da...@lookin2.com> wrote:
>> As our launch date approaches, I am getting increasingly nervous about
>> Cassandra tuning. It is a mysterious black art that I haven't mastered even
>> at the low usages that we have now. I know of a few more things I can do to
>> improve things, but how will I know if it is enough? All this is
>> particularly ironic since - as we are just starting out - we don't have
>> scalability problems yet, though we hope to!
>>
> How are your load tests looking?  Of course, there's nothing like
> going live, but I expect you'll be able to simulate 2x-3x your initial
> launch traffic.
>
>> Luckily, I have completely wrapped Cassandra in an entity mapper, so that I
>> can easily trade in something else, perhaps temporarily, until we really
>> need Cassandra's scalability.
>>
>> So, I'm thinking of emulating Cassandra with mySQL. I would use mySQL either
>> as a simple key-value store, without joins, or map Cassandra supercolumns to
>> mySQL columns, probably of type CLOB.
>>
>> Does anyone want to talk me out of this?
>>
>
> As you said, I think you just have some cold feet.
>
> My feeling is that you did some original research and decided on
> Cassandra for various reasons.  I think if you put the MySQL solution
> in now, you won't go back to the Cassandra solution, because once its
> live, it will be much riskier to switch.  And if you feel you made a
> mistake in your original assessment, then great, at least you found
> out before launch.
>
> Whatever you choose, I would flesh out my my fears with as much detail
> as possible.  Invest in load tests and develop contingency plans.  I
> talked about this in 2009 a little bit here - see slide 22, we call
> these "Defcon Levels".
>
> http://www.slideshare.net/supertom/building-configurable-applications-for-the-web
>
> The idea is prioritizing what REALLY is important if the shit hits the
> fan (watch out, biz folks think everything is always important) and
> having processes to implemen and knobs to turn and levers to pull
> should you get slashdotted (or "facebooked", "tweeted", "oprahed",
> "techcrunched" or whatever we call it these days).
>
> Good luck with your launch.
>
> Thanks,
>
> Tom
>

You should always worry about everything, but you should also have
confidence in your decisions. If your worry is how your cluster will
perform under load, then you should find a way to test under load.
Tweeks and tunes do not make scalability (they help), hardware does.
If you want to be ready to be 'slashdotted' you better have a rack of
servers idling.

If you just need a key-value store you may not need Cassandra.
Cassandra is scalable in a different way then MySQL would be.

You want convincing... (Ill try)
Cassandra shards through node joins and handles replication. If you
start off with a Mysql master/slave architecture, or using id
hash(key) mod 3. It is not clear how you grow that cluster with
demand.

If you make a choice that is not scalable, when you get 'slashdotted'
you will not be ready. What is worse you will have no easy way out of
the problem.

Re: Using mySQL to emulate Cassandra

[Tom Melendez <to...@supertom.com>]

On Sun, Nov 28, 2010 at 12:28 AM, David Boxenhorn <da...@lookin2.com> wrote:
> As our launch date approaches, I am getting increasingly nervous about
> Cassandra tuning. It is a mysterious black art that I haven't mastered even
> at the low usages that we have now. I know of a few more things I can do to
> improve things, but how will I know if it is enough? All this is
> particularly ironic since - as we are just starting out - we don't have
> scalability problems yet, though we hope to!
>
How are your load tests looking?  Of course, there's nothing like
going live, but I expect you'll be able to simulate 2x-3x your initial
launch traffic.

> Luckily, I have completely wrapped Cassandra in an entity mapper, so that I
> can easily trade in something else, perhaps temporarily, until we really
> need Cassandra's scalability.
>
> So, I'm thinking of emulating Cassandra with mySQL. I would use mySQL either
> as a simple key-value store, without joins, or map Cassandra supercolumns to
> mySQL columns, probably of type CLOB.
>
> Does anyone want to talk me out of this?
>

As you said, I think you just have some cold feet.

My feeling is that you did some original research and decided on
Cassandra for various reasons.  I think if you put the MySQL solution
in now, you won't go back to the Cassandra solution, because once its
live, it will be much riskier to switch.  And if you feel you made a
mistake in your original assessment, then great, at least you found
out before launch.

Whatever you choose, I would flesh out my my fears with as much detail
as possible.  Invest in load tests and develop contingency plans.  I
talked about this in 2009 a little bit here - see slide 22, we call
these "Defcon Levels".

http://www.slideshare.net/supertom/building-configurable-applications-for-the-web

The idea is prioritizing what REALLY is important if the shit hits the
fan (watch out, biz folks think everything is always important) and
having processes to implemen and knobs to turn and levers to pull
should you get slashdotted (or "facebooked", "tweeted", "oprahed",
"techcrunched" or whatever we call it these days).

Good luck with your launch.

Thanks,

Tom

Re: Using mySQL to emulate Cassandra

[André Fiedler <fi...@googlemail.com>]

If you want to do this, don´t use autoincrementing integers! ;o) Use UUID
instead. (or implement an stand alone id-server that handles autoincrement
savely)


2010/11/28 David Boxenhorn <da...@lookin2.com>

> As our launch date approaches, I am getting increasingly nervous about
> Cassandra tuning. It is a mysterious black art that I haven't mastered even
> at the low usages that we have now. I know of a few more things I can do to
> improve things, but how will I know if it is enough? All this is
> particularly ironic since - as we are just starting out - we don't have
> scalability problems yet, though we hope to!
>
> Luckily, I have completely wrapped Cassandra in an entity mapper, so that I
> can easily trade in something else, perhaps temporarily, until we really
> need Cassandra's scalability.
>
> So, I'm thinking of emulating Cassandra with mySQL. I would use mySQL
> either as a simple key-value store, without joins, or map Cassandra
> supercolumns to mySQL columns, probably of type CLOB.
>
> Does anyone want to talk me out of this?
>