You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Ian Holsman <ia...@holsman.net> on 2009/11/06 03:13:02 UTC

using cassandra as a real time DW

hey guys.
I was wondering if anyone is thinking of/is using cassandra to power a  
real time data warehouse.
if so would you consider collaborating/open sourcing the effort so  
others could join in.

TIA
Ian.
--
Ian Holsman
Ian@Holsman.net




Re: using cassandra as a real time DW

Posted by Mark Robson <ma...@gmail.com>.
2009/11/6 Joe Stump <jo...@joestump.net>

> On Nov 6, 2009, at 2:35 PM, Mark Robson wrote:
>
> 2009/11/6 Joe Stump <jo...@joestump.net>
>
>>
>> Can you explain what you mean by lack of load balancing?
>>
>
>
> Nothing in Cassandra attempts to ensure that your data are equally spread
> over the different nodes (yet; there are several bugs open to this effect).
>
>
> That's not true from my understanding. It won't put three copies on the
> same node. The key word, I suppose, is "equally".
>

The three copies will generally be on the three nodes sequentially in the
ring, starting at the one nearest to the key.

However, if you have a range of keys that goes from 0000 to 004f say, and
your nodes have tokens 0,2,4,6,8,a,c and e, then you won't get an even
distribution, instead all the data will sit entirely on the first three
nodes with the others completely empty.

It doesn't know to space the tokens evenly throughout the key space. It also
won't change the token of an existing node (Bootstrap can insert new nodes
into the ring and copy / prune the data as necessary, which is a Good
Thing).

You *can* manually assign the tokens and that can be used as a work-around,
if you know what the distribution of your tokens is or is likely to be.

You can also construct your keys carefully such that the tokens are likely
to be equally spaced within them (e.g. by using a hash of something for the
first part of your key).

Other clustered databases (e.g. Hadoop-based things possibly?) split the
data into chunks which then get distributed among the nodes on some
load-balanced basis; Cassandra does not do this yet.

Mark

Re: using cassandra as a real time DW

Posted by Michael Greene <mi...@gmail.com>.
I didn't mean reporting tools in the sense of 'management and  
monitoring tools'. Typically data warehouse platforms have integration  
with presentational report tools, visualization, etc... Things end- 
users can use to query their data. Cassandra is all DIY in this  
category.

Ian: if it's going to end up more like a trading desk, then Cassandra  
may be precisely the tool for this real-time DW, in my experience with  
precisely that application.


On Nov 6, 2009, at 5:20 PM, Joe Stump <jo...@joestump.net> wrote:

>
> On Nov 6, 2009, at 4:16 PM, Ian Holsman wrote:
>
>> Yes the reporting tools are another issue. But I don't think there  
>> is anything that can do rt.
>> This may end up looking more like a trading desk than a standard dw.
>
> We have a patch that exposes a lot of internal metric tracking (and  
> the nodeprobe commands) over Thrift, which was rejected (something  
> about exposing JMX data; I dunno) along with a Munin plugin.
>
> We're happy to share. Hit me up offline if this is of interest.
>
> --Joe

Re: using cassandra as a real time DW

Posted by Joe Stump <jo...@joestump.net>.
On Nov 6, 2009, at 4:16 PM, Ian Holsman wrote:

> Yes the reporting tools are another issue. But I don't think there  
> is anything that can do rt.
> This may end up looking more like a trading desk than a standard dw.

We have a patch that exposes a lot of internal metric tracking (and  
the nodeprobe commands) over Thrift, which was rejected (something  
about exposing JMX data; I dunno) along with a Munin plugin.

We're happy to share. Hit me up offline if this is of interest.

--Joe

Re: using cassandra as a real time DW

Posted by Ian Holsman <ia...@holsman.net>.
Yes the reporting tools are another issue. But I don't think there is  
anything that can do rt.
This may end up looking more like a trading desk than a standard dw.


---
Sent from my phone
Ian Holsman - 703 879-3128

On 07/11/2009, at 9:01 AM, Michael Greene <mi...@gmail.com>  
wrote:

> On Fri, Nov 6, 2009 at 3:46 PM, Joe Stump <jo...@joestump.net> wrote:
>> Nothing in Cassandra attempts to ensure that your data are equally  
>> spread
>> over the different nodes (yet; there are several bugs open to this  
>> effect).
>>
>> That's not true from my understanding. It won't put three copies on  
>> the same
>> node. The key word, I suppose, is "equally".
> Right.  Mark isn't referring to the ReplicationFactor or the
> distribution of an individual piece of data.  He's referring to the
> potential for a series of 100 million rows to all go to the same
> ReplicationFactor count nodes, even if you have a much larger cluster.
> If you use the RandomPartitioner and the various pieces of bootstrap
> functionality in 0.5 or good token picking, this solves the problem.
> If you use the OPP Cassandra is only part of the way there on trunk.
>
>> I think you're misleading people, though, with the notion that a.  
>> Cassandra
>> doesn't have load balancing (it does, in many ways) and b. It  
>> doesn't scale.
> If you are able to tune your data/application and Cassandra to each
> other, it can scale and balance well; I've been very happy with it
> here.  I don't think it is currently usable as a generic data
> warehouse though (in addition to the above, the DIY tooling is a huge
> drawback for someone looking for a generalized DW).
>
> Michael
>
> Michael

Re: using cassandra as a real time DW

Posted by Michael Greene <mi...@gmail.com>.
On Fri, Nov 6, 2009 at 3:46 PM, Joe Stump <jo...@joestump.net> wrote:
> Nothing in Cassandra attempts to ensure that your data are equally spread
> over the different nodes (yet; there are several bugs open to this effect).
>
> That's not true from my understanding. It won't put three copies on the same
> node. The key word, I suppose, is "equally".
Right.  Mark isn't referring to the ReplicationFactor or the
distribution of an individual piece of data.  He's referring to the
potential for a series of 100 million rows to all go to the same
ReplicationFactor count nodes, even if you have a much larger cluster.
 If you use the RandomPartitioner and the various pieces of bootstrap
functionality in 0.5 or good token picking, this solves the problem.
If you use the OPP Cassandra is only part of the way there on trunk.

> I think you're misleading people, though, with the notion that a. Cassandra
> doesn't have load balancing (it does, in many ways) and b. It doesn't scale.
If you are able to tune your data/application and Cassandra to each
other, it can scale and balance well; I've been very happy with it
here.  I don't think it is currently usable as a generic data
warehouse though (in addition to the above, the DIY tooling is a huge
drawback for someone looking for a generalized DW).

Michael

Michael

Re: using cassandra as a real time DW

Posted by Joe Stump <jo...@joestump.net>.
On Nov 6, 2009, at 2:35 PM, Mark Robson wrote:

> 2009/11/6 Joe Stump <jo...@joestump.net>
>
> Can you explain what you mean by lack of load balancing?
>
>
> Nothing in Cassandra attempts to ensure that your data are equally  
> spread over the different nodes (yet; there are several bugs open to  
> this effect).

That's not true from my understanding. It won't put three copies on  
the same node. The key word, I suppose, is "equally".

> If you use the OrderedPartitioner, in all likelihood your data will  
> be very unevenly spread to the point where most of your servers  
> aren't used at all. This obviously doesn't scale.
>
> The RandomPartitioner is better because the hashing it does causes  
> data to spread out, but the tokens are still chosen randomly so  
> there's no way to guarantee that machines get equal or even similar 
> (ish) amounts of data.

We've answered this by creating our own partitioners, which Cassandra  
makes pluggable. Took one of our guys about two full days to have  
something up and running. Also, there's no way to guarantee anything  
for the most part in distributed computing.

I think you're misleading people, though, with the notion that a.  
Cassandra doesn't have load balancing (it does, in many ways) and b.  
It doesn't scale. Digg and Facebook both use it in production and  
while it might not be battle hardened and fully tested, it's  
definitely working for them well under high load.

--Joe


Re: using cassandra as a real time DW

Posted by Mark Robson <ma...@gmail.com>.
2009/11/6 Joe Stump <jo...@joestump.net>

>
> Can you explain what you mean by lack of load balancing?
>


Nothing in Cassandra attempts to ensure that your data are equally spread
over the different nodes (yet; there are several bugs open to this effect).

If you use the OrderedPartitioner, in all likelihood your data will be very
unevenly spread to the point where most of your servers aren't used at all.
This obviously doesn't scale.

The RandomPartitioner is better because the hashing it does causes data to
spread out, but the tokens are still chosen randomly so there's no way to
guarantee that machines get equal or even similar(ish) amounts of data.

Mark

Re: using cassandra as a real time DW

Posted by Joe Stump <jo...@joestump.net>.
On Nov 6, 2009, at 2:17 PM, Ian Holsman wrote:

>> As far as I can see the main issues are the lack of a range_delete  
>> operation and lack of load balancing capability. I can't use  
>> Cassandra right now but will continue to evaluate it in the future.

Can you explain what you mean by lack of load balancing?

--Joe

Re: using cassandra as a real time DW

Posted by Ian Holsman <ia...@holsman.net>.
Thanks Mark.

On Nov 6, 2009, at 10:25 PM, Mark Robson wrote:

> 2009/11/6 Ian Holsman <ia...@holsman.net>
> hey guys.
> I was wondering if anyone is thinking of/is using cassandra to power  
> a real time data warehouse.
> if so would you consider collaborating/open sourcing the effort so  
> others could join in.
>
> As far as I can see the main issues are the lack of a range_delete  
> operation and lack of load balancing capability. I can't use  
> Cassandra right now but will continue to evaluate it in the future.
>
> Mark

--
Ian Holsman
Ian@Holsman.net




Re: using cassandra as a real time DW

Posted by Mark Robson <ma...@gmail.com>.
2009/11/6 Ian Holsman <ia...@holsman.net>

> hey guys.
> I was wondering if anyone is thinking of/is using cassandra to power a real
> time data warehouse.
> if so would you consider collaborating/open sourcing the effort so others
> could join in.
>

As far as I can see the main issues are the lack of a range_delete operation
and lack of load balancing capability. I can't use Cassandra right now but
will continue to evaluate it in the future.

Mark