You are viewing a plain text version of this content. The canonical link for it is here.
Posted to xindice-users@xml.apache.org by "webhiker@tiscali.fr" <we...@tiscali.fr> on 2003/09/17 09:49:03 UTC
Stress testing
Hi,
I've been comparing Xindice and eXist to find out if either of them can
handle the type of usage we require for our project.
For example, maximum number of documents in a single Collection.
Given the simple document :
<test><data attr1="att1" attr2="attr2"/></test>
Xindice 1.0 seems to run out of memory or other resources at about 200
000, while eXist (running in Tomcat) runs out of memory and performance
at about 50 000
documents.
I'm using default, out the box config for both DB's.
I'd appreciate any feedback on similar tests anyone else may have done.
Our project requires we be able to store approximately 500 000
documents, little to no indexing is required.
Does anyone have any ideas of any other DB which will allow us to store
this amount of Documents?
I was under the impression this is not an unreasonable amount of
documents to have in a Collection, but seems to be way off the scale for
these two DB's.
I suppose my alternative is to write a simple file-based DB which stores
each document to disk.
Thanks
WH
Re: Stress testing
Posted by Vadim Gritsenko <va...@verizon.net>.
webhiker@tiscali.fr wrote:
> Hi,
>
> I've been comparing Xindice and eXist to find out if either of them
> can handle the type of usage we require for our project.
> For example, maximum number of documents in a single Collection.
> Given the simple document :
> <test><data attr1="att1" attr2="attr2"/></test>
>
> Xindice 1.0 seems to run out of memory or other resources at about 200
> 000, while eXist (running in Tomcat) runs out of memory and
> performance at about 50 000
> documents.
Xindice 1.0 uses CORBA for client / server communication, and is known
to have some memory leaks (don't know were exactly). I wonder what
results of your testing will be against currect CVS of the xindice.
PS You may want to turn off metadata in config/system.xml if you don't
need it. It will save CPU cycles and disk space
Vadim
> I'm using default, out the box config for both DB's.
> I'd appreciate any feedback on similar tests anyone else may have done.
>
> Our project requires we be able to store approximately 500 000
> documents, little to no indexing is required.
> Does anyone have any ideas of any other DB which will allow us to
> store this amount of Documents?
> I was under the impression this is not an unreasonable amount of
> documents to have in a Collection, but seems to be way off the scale for
> these two DB's.
>
> I suppose my alternative is to write a simple file-based DB which
> stores each document to disk.
>
> Thanks
> WH
Re: Stress testing
Posted by Stan Pinte <st...@axone.be>.
http://www.sleepycat.com/products/xml.shtml
They claim
>Databases up to 256 terabytes
>
>Berkeley DB uses 48 bits to address individual bytes in a database. This
>means that the largest theoretical Berkeley DB database is 248 bytes, or
>256 terabytes, in size. Berkeley DB is in regular production use today
>managing databases that are hundreds of gigabytes in size.
At 09:49 17/09/2003, webhiker@tiscali.fr wrote:
>Content-Transfer-Encoding: 7bit
>
>Hi,
>
>I've been comparing Xindice and eXist to find out if either of them can
>handle the type of usage we require for our project.
>For example, maximum number of documents in a single Collection.
>Given the simple document :
><test><data attr1="att1" attr2="attr2"/></test>
>
>Xindice 1.0 seems to run out of memory or other resources at about 200
>000, while eXist (running in Tomcat) runs out of memory and performance at
>about 50 000
>documents.
>I'm using default, out the box config for both DB's.
>I'd appreciate any feedback on similar tests anyone else may have done.
>
>Our project requires we be able to store approximately 500 000 documents,
>little to no indexing is required.
>Does anyone have any ideas of any other DB which will allow us to store
>this amount of Documents?
>I was under the impression this is not an unreasonable amount of documents
>to have in a Collection, but seems to be way off the scale for
>these two DB's.
>
>I suppose my alternative is to write a simple file-based DB which stores
>each document to disk.
>
>Thanks
>WH
>
>
>
>
>--
>Incoming mail is certified Virus Free.
>Checked by AVG Anti-Virus (http://www.grisoft.com).
>Version: 7.0.167 / Virus Database: 260.1.0 - Release Date: 11/09/2003
--
Outgoing mail is certified Virus Free.
Checked by AVG Anti-Virus (http://www.grisoft.com).
Version: 7.0.167 / Virus Database: 260.1.0 - Release Date: 11/09/2003