You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Dodong Juan <do...@gmail.com> on 2010/05/26 20:45:16 UTC
Doing joins between column familes
So I am not sure if you guys are familiar with OCM . Basically it is
an ORM for Cassandra. Been testing it
So I have created model that has the following object relationship.
OCM generates the code from this that allows me to do easy
programmatic query from Java to Cassandra.
Object1-(Many2Many)->Object2-(Many2Many)->Object3-(Many2Many)->Object4-
(Many2Many)->Node
So my app gets the NODE and tries to query the dependency relationship
from Node->Object4->Object3->Object2->Object1.
I have compared the performance between Cassandra(with OCM) vs DB2.
The result is not very encouraging since the DB2 performance is
showing at least 3X faster than Cassandra. DB2 basically is just a
single call with a number of inner joins ..
Looking at the code, I think we might get a better performance if
somehow we can do the joins between objects within Cassandra server
rather than the client side. Right now , I am basically doing the
following.
Node node = new Node(connection,"nodeidentifier");
node.loadInfo(); // going to the wire ...?
node.loadObject4(); // this goes to the wire too ..
object4Keys = node.getObject4().getColumns.keys();
while(object4Keys.hasNextElement)
{
object4Key = object4Key.nextElement();
object4 = node.getObject4().get(object4Key);
object4.loadInfo(); // this goes to the wire
too ..
object4.loadObject3(); // this goes to the wire too ..
object3Keys = object4.getObject3().getColumns.keys();
while(object3Keys.hasNextElement)
{
object3Key = object3Key.nextElement();
object3 = node.getObject4().get(object4Key);
object3.loadInfo(); // this goes to the wire too ..
object3.loadObject2(); // this goes to the wire too ..
...
..
... until you get object1//
}
I think there is a lot of going back and forth between the client and
cassandra and if we can only move the relationship joins to Cassandra
server I think we can minimize the latency and improve the overall
performance on the query.
Is there a way to do a join across ColumnFamilies in Cassandra ?
Re: Doing joins between column familes
Posted by Jonathan Shook <js...@gmail.com>.
I wrote some Iterable<*> methods to do this for column families that
share key structure with OPP.
It is on the hector examples page. Caveat emptor.
It does iterative chunking of the working set for each column family,
so that you can set the nominal transfer size when you construct the
Iterator/Iterable. I've been very happy with the performance of it,
even over large ranges of keys. This is with
OrderPreservingPartitioner because of other requirements, so it may
not be a good example for comparison with a random partitioner, which
is preferred.
Doing joins as such on the server works against the basic design of
Cassandra. The server does a few things very well only because it
isn't overloaded with extra faucets and kitchen sinks. However, I'd
like to be able to load auxiliary classes into the server runtime in a
modular way, just for things like this. Maybe we'll get that someday.
My impression is that there is much more common key structure in a
workable Cassandra storage layout than in a conventional ER model.
This is the nature of the beast when you are organizing your
information more according to access patterns than fully normal
relationships. That is one of the fundamental design trade-offs of
using a hash structure over a schema.
Having something that lets you deploy a fully normal schema on a hash
store can be handy, but it can also obscure the way that your
application indirectly exercises the storage layer. The end-result may
be that the layout is less friendly to the underlying mechanisms of
Cassandra. I'm not saying that it is bad to have a tool to do this,
only that it can make it easy to avoid thinking about Cassandra
storage in terms of what it really is.
There may be ways to optimize the OCM queries, but that takes you down
the road of query optimization, which can be quite nebulous. My gut
instinct is to focus more on the layout, using aggregate keys and
common key structure where you can, so that you can take advantage of
the parallel queries more of the time.
On Wed, May 26, 2010 at 3:13 PM, Charlie Mason <ch...@gmail.com> wrote:
> On Wed, May 26, 2010 at 7:45 PM, Dodong Juan <do...@gmail.com> wrote:
>>
>> So I am not sure if you guys are familiar with OCM . Basically it is an ORM
>> for Cassandra. Been testing it
>>
>
> In case anyone is interested I have posted a reply on the OCM issue
> tracker where this was also raised.
>
> http://github.com/charliem/OCM/issues/closed#issue/5/comment/254717
>
>
> Charlie M
>
Re: Doing joins between column familes
Posted by Charlie Mason <ch...@gmail.com>.
On Wed, May 26, 2010 at 7:45 PM, Dodong Juan <do...@gmail.com> wrote:
>
> So I am not sure if you guys are familiar with OCM . Basically it is an ORM
> for Cassandra. Been testing it
>
In case anyone is interested I have posted a reply on the OCM issue
tracker where this was also raised.
http://github.com/charliem/OCM/issues/closed#issue/5/comment/254717
Charlie M