Poor Man's Clustered DataSource Package.

[Leif Mortenson <le...@silveregg.co.jp>]

 We have been working on scalability issues for an application that we
have implented on Apache. One of the big bottlenecks has been our
backend database. We needed to have a way to distribute our data without
having to pay for a truely clustered database solution.

We decided to implement a kind of poor man's cluster by dividing our
data amongst a cluster of backend databases and then accessing the data
using the modulous of the hash of the data's primary key. This lets us
reliably access the data even though it is distributed across several
databases.

I added a new package called cluster under the datasource package in the
scratchpad containing the classes that we are using. It contains 3
implementations HashedDataSourceCluster.java,
IndexedDataSourceCluster.java, and RoundRobinDataSourceCluster.java. The
Javadocs for each class explain how to get up and running with them.
Basicly, they each allow you to define a set of regular DataSources
which will then be treated as a DataSource Cluster. You modify the
configuration of your user component so that the dbpool tag points to a
Cluster, which implements Cluster, rather than a DataSource. The
RoundRobinDataSourceCluster will work without any modifications to your
code, but it has limited applications. The HashedDataSourceCluster
requires that you modify calls to getConnection so that they pass in a
hashCode or an index, in the case of IndexedDataSourceCluster. (They are
each described in more depth in the javadocs)

I was wondering if anyone on the list has had any experience setting up
Data Base Clusters like this who might have some advice on how they
could be better implemented. These have been working for us so far, but
are still new.

Cheers,
Leif


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Poor Man's Clustered DataSource Package.

[Peter Donald <pe...@apache.org>]

Hi,

On Mon, 4 Feb 2002 14:39, Leif Mortenson wrote:
> I was wondering if anyone on the list has had any experience setting up
> Data Base Clusters like this who might have some advice on how they
> could be better implemented. These have been working for us so far, but
> are still new.

I just went through commitlog and looks good. I haven't done a lot of JDBC 
work in that area so I can't offer any suggestions but I got to say I like 
the way you code. There is almost more documentation than there is code!!! ;)

-- 
Cheers,

Pete

-------------------------------------------------------
"When we remember we are all mad, the mysteries of life 
disappear and life stands explained." -Mark Twain
-------------------------------------------------------

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: Poor Man's Clustered DataSource Package.

[Berin Loritsch <bl...@apache.org>]

Leif Mortenson wrote:

>  We have been working on scalability issues for an application that we
> have implented on Apache. One of the big bottlenecks has been our
> backend database. We needed to have a way to distribute our data without
> having to pay for a truely clustered database solution.
> 
> We decided to implement a kind of poor man's cluster by dividing our
> data amongst a cluster of backend databases and then accessing the data
> using the modulous of the hash of the data's primary key. This lets us
> reliably access the data even though it is distributed across several
> databases.


Awesome!


<snip/>


> I was wondering if anyone on the list has had any experience setting up
> Data Base Clusters like this who might have some advice on how they
> could be better implemented. These have been working for us so far, but
> are still new.


Maybe not, but as you can see with the DataSource code in general, it just
takes a few iterations before we can see true potential for improving the
existing design.




-- 

"They that give up essential liberty to obtain a little temporary safety
 deserve neither liberty nor safety."
                - Benjamin Franklin


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>