You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Olav Sandstaa <ol...@sun.com> on 2006/12/01 00:14:55 UTC
Re: Blog response to Oracle white paper
Great blog contribution, David! I also find the report highly biased
and agree with the other comments
regarding the presented results and the conclusion that is made. But
even if comparing Derby to BDB Java edition is comparing apples to
oranges, still I think some of the results are worth to look a bit
into.
For instance, the first reference to this report was sent on a thread
on derby-user where the topic of the thread was someone who wanted to
use the Derby store directly to avoid using "SQL/JDBC because it has
unacceptable overhead" and where the main access functions were
put/get of key based data [1]. As response he got advices ranging from
"use BDB Java edition" to "try it with Derby to determine if the
overhead of using JDBC/SQL indeed is too high".
Several places in the BDB paper the same conclusion is reached that
the lower performance of Derby compared to BDB is "probably due to
the overhead of SQL processing in Derby". The experiment that is most
similar to the "get record based on key" as requested in [1] is shown
in the figure named "Random read". In this figure it seems like BDB has 5X
the performance of Derby (100.000 vs 20.000 records per second).
I think it would be useful if we understood the main reasons for why
BDB is able to retrieve five times more tuples than Derby. I find it
hard to believe that the "overhead of SQL processing" given that
prepared statements are used should add 400% to the cost of
"retrieving a record from a b-tree". I do not know BDB Java edition
well, but as far as I understand it is running with transactions and
transaction isolation so that should be a cost to both BDB and
Derby. I have not seen the code for the test application(s) so it is a
bit hard to know what they actually measure, but the 20.000
records/second for Derby seems similar to the number of single-record
select queries I get when running on comparable hardware.
Are there other necessary functionality present in Derby (and missing
in BDB) that is adding to the cost besides the JDBC and SQL layer? Or
is most of these 400% extra cost mostly contributed by "overhead of
SQL processing"? It would be great to get opinions from other on the
list on both if this is a reasonable overhead compared to BDB and what
are the causes for this overhead, and whether this is something that
we should try to improve. It could of course also be that BDB
is doing something either "very smart" or that the test that has been
run is not fair in some way.
Thanks,
Olav
[1] http://www.nabble.com/Using-Derby-as-a-binary-store-tf2696662.html
Re: Blog response to Oracle white paper
Posted by Daniel John Debrunner <dj...@apache.org>.
Olav Sandstaa wrote:
>
> Several places in the BDB paper the same conclusion is reached that
> the lower performance of Derby compared to BDB is "probably due to
> the overhead of SQL processing in Derby". The experiment that is most
> similar to the "get record based on key" as requested in [1] is shown
> in the figure named "Random read". In this figure it seems like BDB has 5X
> the performance of Derby (100.000 vs 20.000 records per second).
I actually think this is the 10x measurement that justifies the top-end
of the claim of being 10x slower. Apart from not having the source I
can't see any description of "random read" in the paper. So there's a
graph that shows Derby being 10x slower with absolutely zero explanation
for what was being measured.
I think it would be useful to setup a standard benchmark, based around
these types of applications that could be implemented against
BerkeleyDB, Hibernate, SQL, JDO, JPA etc. That is the benchmark would be
defined in abstract terms, that would allow multiple implementations.
Then at least we could all have some confidence over what was being
measured.
I agree with you & Suresh, that 400% or 1000% seems too much for SQL
processing, but without seeing if the applications are receiving
identical functionality from the databases it hard to see what is going
on. One possible difference is the byte[] array returned from the api.
With JDBC the value returned is private to the application and thus the
application can modify it without making a copy. Is the same true for
BerkeleyDB, or does its api allow it to return a single value that the
application needs to copy if it wants to modify?
Dan.
Re: Blog response to Oracle white paper
Posted by Suresh Thalamati <su...@gmail.com>.
Olav Sandstaa wrote:
> Great blog contribution, David! I also find the report highly biased
<snip ..>
> Are there other necessary functionality present in Derby (and missing
> in BDB) that is adding to the cost besides the JDBC and SQL layer? Or
> is most of these 400% extra cost mostly contributed by "overhead of
> SQL processing"? It would be great to get opinions from other on the
> list on both if this is a reasonable overhead compared to BDB and what
> are the causes for this overhead, and whether this is something that
> we should try to improve. It could of course also be that BDB
> is doing something either "very smart" or that the test that has been
> run is not fair in some way.
>
> Thanks,
> Olav
>
> [1] http://www.nabble.com/Using-Derby-as-a-binary-store-tf2696662.html
>
I agree with you 400% overhead due to SQ for simple queries is
unlikely. I wonder if DB is storing both key and values in one file.
In derby key (in tree) and values (in heap) are getting stored in
different files , may be this is causing an extra I/O in Derby for
this test.
Thanks
-surest