You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@labs.apache.org by ka...@apache.org on 2009/03/18 20:10:54 UTC
svn commit: r755705 - /labs/bananadb/trunk/src/site/apt/index.apt
Author: kalle
Date: Wed Mar 18 19:10:53 2009
New Revision: 755705
URL: http://svn.apache.org/viewvc?rev=755705&view=rev
Log:
BananaDB
website
Modified:
labs/bananadb/trunk/src/site/apt/index.apt
Modified: labs/bananadb/trunk/src/site/apt/index.apt
URL: http://svn.apache.org/viewvc/labs/bananadb/trunk/src/site/apt/index.apt?rev=755705&r1=755704&r2=755705&view=diff
==============================================================================
--- labs/bananadb/trunk/src/site/apt/index.apt (original)
+++ labs/bananadb/trunk/src/site/apt/index.apt Wed Mar 18 19:10:53 2009
@@ -1,24 +1,30 @@
-----
embedded key/value pair database for java
-BananaDB
+Banana DB
- BananaDB is a small (100KB jar) embedded byte array key/value pair database for Java
- featuring a top level API similar to working with any Map\<K, V\>
-
+ Banana DB is an embedded byte[] key/value pair database implemented in Java.
-* Main features
+
+* Features
+
+ * Small, ~100KB .jar file with no dependencies
+
+ * Top level API similar to working with any Map\<K, V\>
* Thread safe and write locking over multiple JVMs
- * Atomic access
+ * Optionally transactional
+
+ * Not ACID (only the I-part)
- * Transactional access with multiple isolation level strategies
+ * Multiple isolation level strategies in transactional mode
* Annotational API
* Pluggable serialization strategies (java.io.Serialization as default)
+
* Upcoming features
@@ -29,41 +35,89 @@
* Optional incrementing sequences for key postings
+ * ACID
-* Performance and limitations
+Releases
- * Relies heavy on media I/O seek, i.e. SSD and file cache is helpfull.
+ There is no official release! You'll have to check it out from SVN, compile and install:
- * Benchmarks on 5400RPM HDD MacBook shows that
++-------------------------------+
- * Adding new <byte[4], byte[random(1-5000)> postings takes about
- 1 millisecond + (1 millisecond per 500,000 postings in DB)
+limax:bananadb kalle$ svn co http://svn.apache.org/repos/asf/labs/bananadb/trunk/
+A trunk/FILES.txt
+A trunk/LICENSE.txt
+..
- * Retrieving the
+limax:bananadb kalle$ cd trunk
+limax:trunk kalle$ mvn install
- * No read speed penalty due to deleting or updating key/value postings
++-------------------------------+
- * No limit on number of key/value postings in database.
+Performance and limitations
+ See benchmark further down in this document.
-* Releases
+ Banana DB use an ad hoc data format that does not try to optimize the distance between
+ postings. This means that the speed is very much depending on I/O seek of the
+ storage media and the file cache of the operative system.
- There is no official release! You'll have to check it out from SVN, compile and install:
+ The hypothesis is that SSD is now a cheap medium that eliminates the need for
+ more complex data formats such as B+tree or HTree.
-+-------------------------------+
+ There is no issue with read and write speed for small and medium sized databases
+ located on HDD with a modern operative system that does smart file caching.
-limax:bananadb kalle$ svn co http://svn.apache.org/repos/asf/labs/bananadb/trunk/
-A trunk/FILES.txt
-A trunk/LICENSE.txt
-..
-limax:bananadb kalle$ cd trunk
-limax:trunk kalle$ mvn install
+ * No limit on number of key/value postings in database
-+-------------------------------+
+ * Up to 180 petabyte of value postings data and 140 petabyte of key postings data
+
+ * No read speed penalty due to deleted or updated key/value postings
+
+
+Compared to other databases
+
+ Banana DB is pretty much the same thing as many other implementations.
+ The reason for Banana DB to exist is the lack of a key/value database in Java distributed with the Apache Software Licene.
+
+ I am not a lawyer. Please double check any information regaarding licening models mentioned below.
+
+
+* Berkeley DB
+
+ {{{http://www.oracle.com/database/berkeley-db.html}http://www.oracle.com/database/berkeley-db.html}}
+
+ Berkeley DB and Banana DB is more or less the same thing if you leave out fact they
+ use completly different underlying file formats. Berkeley DB is also ACID compliant
+ which Banana DB is not.
+ The Banana DB entity store API is almost identical to the Berkeley DB Java Edition API.
+
+ Berkely DB has a dual licene model.
+
+ The license model for propriatary projects
+ allows you to use Berkeley DB at no cost
+ when used in a service you provide,
+ but not if your software is sold as a boxed product.
+
+ The license model for open source projects
+ allows you to redistribute Berkeley DB.
+
+
+* JDBM
+
+ {{{http://jdbm.sourceforge.net/}http://jdbm.sourceforge.net/}}
+
+ JDBM has similar features as Banana DB
+ but a more complex user API compared to Banana DB.
+ It also has support for pluggable storage formats,
+ including B+Tree and HTree storage.
+
+ It is licensed with BSD open source license.
+
+
API examples
@@ -135,3 +189,148 @@
+-------------------------------+
+Benchmarks
+
+ org.apache.labs.bananadb.Benchmark takes the following arguments:
+
+ * Minimum value posting byte[] size
+
+ * Maximum value posting byte[] size
+
+ * Number of postings
+
+ * Debug interval
+
+
+ This is what it does:
+
+ 1. Creates all postings.
+ byte[4] (java.lang.Long) key and random sized value according to your settings.
+
+ 2. Reads all postings in random order.
+
+ 3. Reads all postings in reversed chronlogical order.
+ I.e. the postings that requires maximum number of seeks in DB is executed first.
+
+
+ Below is the output from 1M posting benchmark on a 2GB RAM, 2Ghz DualCore, 5400 RPM HDD, running OpenJDK on Ubuntu Linux.
+
+ Notice that writing is much more efficient than reading randomly.
+ This is because the file cache at write time contains all parts that needs to be seeked and written to,
+ while during random read time the values partition (5GB) is more or less never in file cache.
+
+ You can expect this to be true even for applications busy read and writing at the same time
+ as value postings file is written to in chronological order (i.e. very simple file format)
+ and should thus be easy for the operative system to keep cached.
+
++------------------------------+
+
+limax:trunk kalle$ mvn install
+limax:trunk kalle$ java -cp target/bananadb.jar org.apache.labs.bananadb.Benchmark 1 10000 1000000 10000
+
+1237382243269 INFO Benchmark: Arguments: [min value posting byte[] size] [max value posting byte[] size] [number of postings] [debug interval]
+1237382243286 INFO Benchmark: Random seed: 1237382243286
+1237382243287 INFO Benchmark: Benchmarking store. minValueSize=1, maxValueSize=10000, benchmarkSize=1000000, debugInterval=10000
+1237382245256 INFO Accessor: Creating new store..
+1237382245257 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.md..
+1237382245258 INFO FileHandler: It took 1 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.md
+1237382245259 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.ht..
+1237382245608 INFO FileHandler: It took 349 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.ht
+1237382245610 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.hc..
+1237382246249 INFO FileHandler: It took 638 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.hc
+1237382246250 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.k..
+1237382247584 INFO FileHandler: It took 1334 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.k
+1237382247585 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.v..
+1237382253025 INFO FileHandler: It took 5440 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000000.v
+1237382253276 INFO Accessor: New store has been created. Took 8020 milliseconds.
+1237382253276 INFO Benchmark: Starting benchmark..
+1237382261851 INFO Benchmark: Wrote 10000 postings in 8 seconds or 0.857 ms/posting. 0.856 ms/posting since last report.
+1237382271343 INFO Benchmark: Wrote 20000 postings in 18 seconds or 0.903 ms/posting. 0.949 ms/posting since last report.
+1237382272212 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000001.v..
+1237382273825 INFO FileHandler: It took 1612 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000001.v
+1237382282682 INFO Benchmark: Wrote 30000 postings in 29 seconds or 0.980 ms/posting. 1.134 ms/posting since last report.
+1237382292123 INFO Benchmark: Wrote 40000 postings in 38 seconds or 0.971 ms/posting. 0.944 ms/posting since last report.
+
+...
+
+1237383003175 INFO Benchmark: Wrote 470000 postings in 749 seconds or 1.596 ms/posting. 2.692 ms/posting since last report.
+1237383019220 INFO Benchmark: Wrote 480000 postings in 765 seconds or 1.596 ms/posting. 1.604 ms/posting since last report.
+1237383022560 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000023.v..
+1237383024617 INFO FileHandler: It took 2057 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000023.v
+1237383046559 INFO Benchmark: Wrote 490000 postings in 793 seconds or 1.619 ms/posting. 2.734 ms/posting since last report.
+1237383065630 INFO Benchmark: Wrote 500000 postings in 812 seconds or 1.625 ms/posting. 1.907 ms/posting since last report.
+1237383070151 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000024.v..
+1237383072161 INFO FileHandler: It took 2010 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000024.v
+1237383088374 INFO Benchmark: Wrote 510000 postings in 835 seconds or 1.637 ms/posting. 2.274 ms/posting since last report.
+1237383112307 INFO Benchmark: Wrote 520000 postings in 859 seconds or 1.652 ms/posting. 2.393 ms/posting since last report.
+...
+1237383608557 INFO Benchmark: Wrote 740000 postings in 1355 seconds or 1.831 ms/posting. 2.318 ms/posting since last report.
+1237383628946 INFO Benchmark: Wrote 750000 postings in 1375 seconds or 1.834 ms/posting. 2.039 ms/posting since last report.
+1237383638670 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000036.v..
+1237383640570 INFO FileHandler: It took 1900 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000036.v
+1237383657785 INFO Benchmark: Wrote 760000 postings in 1404 seconds or 1.848 ms/posting. 2.884 ms/posting since last report.
+1237383676019 INFO Benchmark: Wrote 770000 postings in 1422 seconds or 1.848 ms/posting. 1.823 ms/posting since last report.
+...
+1237384190548 INFO Benchmark: Wrote 980000 postings in 1937 seconds or 1.977 ms/posting. 2.010 ms/posting since last report.
+1237384198252 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000047.v..
+1237384199390 INFO FileHandler: It took 1138 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000047.v
+1237384214242 INFO Benchmark: Wrote 990000 postings in 1960 seconds or 1.981 ms/posting. 2.369 ms/posting since last report.
+1237384234525 INFO Benchmark: Wrote 1000000 postings in 1981 seconds or 1.981 ms/posting.
+
+
+
+1237384717428 INFO Benchmark: Begin retreiving postings in random order
+1237384907614 INFO Benchmark: Retreived (random keys) 10000 postings in 190 seconds or 19.018 ms/posting. 19.018 ms/posting since last report.
+1237385036614 INFO Benchmark: Retreived (random keys) 20000 postings in 319 seconds or 15.959 ms/posting. 12.900 ms/posting since last report.
+1237385149594 INFO Benchmark: Retreived (random keys) 30000 postings in 432 seconds or 14.406 ms/posting. 11.298 ms/posting since last report.
+1237385261029 INFO Benchmark: Retreived (random keys) 40000 postings in 543 seconds or 13.590 ms/posting. 11.144 ms/posting since last report.
+1237385373218 INFO Benchmark: Retreived (random keys) 50000 postings in 655 seconds or 13.116 ms/posting. 11.219 ms/posting since last report.
+1237385486088 INFO Benchmark: Retreived (random keys) 60000 postings in 768 seconds or 12.811 ms/posting. 11.287 ms/posting since last report.
+1237385601175 INFO Benchmark: Retreived (random keys) 70000 postings in 883 seconds or 12.625 ms/posting. 11.509 ms/posting since last report.
+...
+1237395567837 INFO Benchmark: Retreived (random keys) 940000 postings in 10850 seconds or 11.543 ms/posting. 11.422 ms/posting since last report.
+1237395682805 INFO Benchmark: Retreived (random keys) 950000 postings in 10965 seconds or 11.543 ms/posting. 11.497 ms/posting since last report.
+1237395796728 INFO Benchmark: Retreived (random keys) 960000 postings in 11079 seconds or 11.541 ms/posting. 11.392 ms/posting since last report.
+1237395910212 INFO Benchmark: Retreived (random keys) 970000 postings in 11192 seconds or 11.539 ms/posting. 11.348 ms/posting since last report.
+1237396023759 INFO Benchmark: Retreived (random keys) 980000 postings in 11306 seconds or 11.537 ms/posting. 11.355 ms/posting since last report.
+1237396137397 INFO Benchmark: Retreived (random keys) 990000 postings in 11419 seconds or 11.535 ms/posting. 11.364 ms/posting since last report.
+1237396251174 INFO Benchmark: Retreived (random keys) 1000000 postings in 11419 seconds or 11.420 ms/posting. 11.378 ms/posting since last report.
+...
+1237395910212 INFO Benchmark: Retreived (random keys) 970000 postings in 11192 seconds or 11.539 ms/posting. 11.348 ms/posting since last report.
+1237396023759 INFO Benchmark: Retreived (random keys) 980000 postings in 11306 seconds or 11.537 ms/posting. 11.355 ms/posting since last report.
+1237396137397 INFO Benchmark: Retreived (random keys) 990000 postings in 11419 seconds or 11.535 ms/posting. 11.364 ms/posting since last report.
+1237396251174 INFO Benchmark: Retreived (random keys) 1000000 postings in 11419 seconds or 11.420 ms/posting. 11.378 ms/posting since last report.
+
+
+1237397304222 INFO Benchmark: Begin retreiving postings in reverse chronological order (i.e. maximum number of seeks in order to get a post first)
+1237397321664 INFO Benchmark: Retreived (backwards chronologial order) 10000 postings in 17 seconds or 1.744 ms/posting. 1.744 ms/posting since last report.
+1237397339067 INFO Benchmark: Retreived (backwards chronologial order) 20000 postings in 34 seconds or 1.742 ms/posting. 1.740 ms/posting since last report.
+1237397356277 INFO Benchmark: Retreived (backwards chronologial order) 30000 postings in 52 seconds or 1.735 ms/posting. 1.721 ms/posting since last report.
+1237397373485 INFO Benchmark: Retreived (backwards chronologial order) 40000 postings in 69 seconds or 1.732 ms/posting. 1.721 ms/posting since last report.
+1237397390635 INFO Benchmark: Retreived (backwards chronologial order) 50000 postings in 86 seconds or 1.728 ms/posting. 1.715 ms/posting since last report.
+1237397407837 INFO Benchmark: Retreived (backwards chronologial order) 60000 postings in 103 seconds or 1.727 ms/posting. 1.720 ms/posting since last report.
+...
+1237398618712 INFO Benchmark: Retreived (backwards chronologial order) 910000 postings in 1314 seconds or 1.444 ms/posting. 1.044 ms/posting since last report.
+1237398628788 INFO Benchmark: Retreived (backwards chronologial order) 920000 postings in 1324 seconds or 1.440 ms/posting. 1.008 ms/posting since last report.
+1237398638361 INFO Benchmark: Retreived (backwards chronologial order) 930000 postings in 1334 seconds or 1.435 ms/posting. 0.957 ms/posting since last report.
+1237398647883 INFO Benchmark: Retreived (backwards chronologial order) 940000 postings in 1343 seconds or 1.429 ms/posting. 0.952 ms/posting since last report.
+1237398657182 INFO Benchmark: Retreived (backwards chronologial order) 950000 postings in 1352 seconds or 1.424 ms/posting. 0.930 ms/posting since last report.
+1237398666443 INFO Benchmark: Retreived (backwards chronologial order) 960000 postings in 1362 seconds or 1.419 ms/posting. 0.926 ms/posting since last report.
+1237398675435 INFO Benchmark: Retreived (backwards chronologial order) 970000 postings in 1371 seconds or 1.414 ms/posting. 0.899 ms/posting since last report.
+1237398684210 INFO Benchmark: Retreived (backwards chronologial order) 980000 postings in 1379 seconds or 1.408 ms/posting. 0.878 ms/posting since last report.
+1237398692941 INFO Benchmark: Retreived (backwards chronologial order) 990000 postings in 1388 seconds or 1.403 ms/posting. 0.873 ms/posting since last report.
+1237398701489 INFO Benchmark: Retreived (backwards chronologial order) 1000000 postings in 1388 seconds or 1.389 ms/posting. 0.855 ms/posting since last report.
+
+
+1237398701489 INFO Benchmark: Rehashing with factor 1.7
+1237398701523 INFO Store: Rehashing hashtable postings capacity from 8,000,000 -> 13,600,000
+1237398701523 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000001.ht..
+1237398703445 INFO FileHandler: It took 1921 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000001.ht
+1237398703446 INFO FileHandler: Formatting /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000001.hc..
+1237398704234 INFO FileHandler: It took 788 milliseconds to format /home/kalle/projekt/bananadb/trunk/benchmark/1237382245206/00000001.hc
+1237399208524 INFO Benchmark: Rehashing took 507 seconds
+1237399208613 INFO Store: Closing store..
+1237399208614 INFO Store: Store has been closed.
+
+
++------------------------------+
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@labs.apache.org
For additional commands, e-mail: commits-help@labs.apache.org