You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Olav Sandstaa <ol...@sun.com> on 2006/12/01 00:14:55 UTC

Re: Blog response to Oracle white paper

Great blog contribution, David!  I also find the report highly biased 
and agree with the other comments
regarding the presented results and the conclusion that is made. But
even if comparing Derby to BDB Java edition is comparing apples to
oranges, still I think some of the results are worth to look a bit
into.

For instance, the first reference to this report was sent on a thread
on derby-user where the topic of the thread was someone who wanted to
use the Derby store directly to avoid using "SQL/JDBC because it has
unacceptable overhead" and where the main access functions were
put/get of key based data [1]. As response he got advices ranging from
"use BDB Java edition" to "try it with Derby to determine if the
overhead of using JDBC/SQL indeed is too high".

Several places in the BDB paper the same conclusion is reached that
the lower performance of Derby compared to BDB is "probably due to
the overhead of SQL processing in Derby". The experiment that is most
similar to the "get record based on key" as requested in [1] is shown
in the figure named "Random read". In this figure it seems like BDB has 5X
the performance of Derby (100.000 vs 20.000 records per second).

I think it would be useful if we understood the main reasons for why
BDB is able to retrieve five times more tuples than Derby. I find it
hard to believe that the "overhead of SQL processing" given that
prepared statements are used should add 400% to the cost of
"retrieving a record from a b-tree". I do not know BDB Java edition
well, but as far as I understand it is running with transactions and
transaction isolation so that should be a cost to both BDB and
Derby. I have not seen the code for the test application(s) so it is a
bit hard to know what they actually measure, but the 20.000
records/second for Derby seems similar to the number of single-record
select queries I get when running on comparable hardware.

Are there other necessary functionality present in Derby (and missing
in BDB) that is adding to the cost besides the JDBC and SQL layer? Or
is most of these 400% extra cost mostly contributed by "overhead of
SQL processing"? It would be great to get opinions from other on the
list on both if this is a reasonable overhead compared to BDB and what
are the causes for this overhead, and whether this is something that
we should try to improve. It could of course also be that BDB
is doing something either "very smart" or that the test that has been
run is not fair in some way.

Thanks,
Olav
 
[1] http://www.nabble.com/Using-Derby-as-a-binary-store-tf2696662.html

Re: Blog response to Oracle white paper

Posted by Daniel John Debrunner <dj...@apache.org>.
Olav Sandstaa wrote:

> 
> Several places in the BDB paper the same conclusion is reached that
> the lower performance of Derby compared to BDB is "probably due to
> the overhead of SQL processing in Derby". The experiment that is most
> similar to the "get record based on key" as requested in [1] is shown
> in the figure named "Random read". In this figure it seems like BDB has 5X
> the performance of Derby (100.000 vs 20.000 records per second).

I actually think this is the 10x measurement that justifies the top-end 
of the claim of being 10x slower. Apart from not having the source I 
can't see any description of "random read" in the paper. So there's a 
graph that shows Derby being 10x slower with absolutely zero explanation 
for what was being measured.

I think it would be useful to setup a standard benchmark, based around 
these types of applications that could be implemented against 
BerkeleyDB, Hibernate, SQL, JDO, JPA etc. That is the benchmark would be 
defined in abstract terms, that would allow multiple implementations. 
Then at least we could all have some confidence over what was being 
measured.

I agree with you & Suresh, that 400% or 1000% seems too much for SQL 
processing, but without seeing if the applications are receiving 
identical functionality from the databases it hard to see what is going 
on. One possible difference is the byte[] array returned from the api. 
With JDBC the value returned is private to the application and thus the 
application can modify it without making a copy. Is the same true for 
BerkeleyDB, or does its api allow it to return a single value that the 
application needs to copy if it wants to modify?

Dan.



Re: Blog response to Oracle white paper

Posted by Suresh Thalamati <su...@gmail.com>.
Olav Sandstaa wrote:
> Great blog contribution, David!  I also find the report highly biased 
<snip ..>
> Are there other necessary functionality present in Derby (and missing
> in BDB) that is adding to the cost besides the JDBC and SQL layer? Or
> is most of these 400% extra cost mostly contributed by "overhead of
> SQL processing"? It would be great to get opinions from other on the
> list on both if this is a reasonable overhead compared to BDB and what
> are the causes for this overhead, and whether this is something that
> we should try to improve. It could of course also be that BDB
> is doing something either "very smart" or that the test that has been
> run is not fair in some way.
> 
> Thanks,
> Olav
> 
> [1] http://www.nabble.com/Using-Derby-as-a-binary-store-tf2696662.html
> 

I agree with you 400% overhead due to SQ for simple queries is 
unlikely. I wonder if DB is storing both key and values in one file.
In derby key (in tree) and values (in heap) are getting stored in 
different files , may  be this is causing an extra I/O in Derby for 
this test.


Thanks
-surest