You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@xmlbeans.apache.org by Peter Schübel <Pe...@gmx.de> on 2007/02/19 19:10:57 UTC

XmlBeans and Databases (db4o)

Hello!

I am working on a project where various application data is initially read in from XML files. This data may later be updated through a remote connection. Via another remote connection, the data is made available to another computer that does further processing. Basically we have several sources that need to be managed. Therefore it is desirable to have a database storing and managing all this data. We decided to try the object-oriented database db4o (www.db4o.com). XML processing (as validation is a must!) is currently done with XMLBeans. It is very handy as we don't have to manually create the 66 classes for all of our 3 Schemas.

I searched the db4o forum and already found out that someone has had problems with XMLBeans. I also ran into a few, but managed to do some workarounds for them. Still there remains a fundamental problem that just makes the use of XMLBeans terribly inefficient in combination with db4o.
I am talking about a (probably xmlbeans-induced) database file size increase of about 30 times compared to a simple example. I'd like to know if anyone can give me some advice on what best to do.


Consider the following code, where i rebuilt the db4o Pilot example with XMLBeans (class PilotType, let's call them PilotBeans):

public static void main(String[] args) {
    Db4o.configure().exceptionsOnNotStorable(true);
    String databaseFilename = new String("Z:\\SVN\\tests\\db4o\\TestSimplePilot.db4o");
    String databaseFilenameXB = new String("Z:\\SVN\\tests\\db4o\\TestXmlBeanPilot.db4o");

    ObjectContainer _db=Db4o.openFile(databaseFilename);
    try {
        clearDatabase(_db);

        System.out.print("Storing simple Objects...\n");
        for(int i=0; i<1000; ++i) {
            _db.set(new Pilot("Michael Schumacher",100));
            _db.set(new Pilot("Rubens Barrichello",99));
        }
        retrieveComplexNQ(_db);
    }
    finally {
        _db.close();
    }
    
    ObjectContainer _dbt=Db4o.openFile(databaseFilenameXB);
    try {
        clearDatabaseXB(_dbt);
    
        System.out.print("Storing XmlBean Objects...\n");
        for(int i=0; i<1000; ++i) {
            PilotType michi = PilotType.Factory.newInstance();
            PilotType rubi  = PilotType.Factory.newInstance();
            michi.setFirstName("Michael");
            michi.setLastName("Schumacher");
            michi.setPoints(new BigInteger("100"));
            rubi.setFirstName("Rubens");
            rubi.setLastName("Barrichello");
            rubi.setPoints(new BigInteger("99"));

            _dbt.set(michi);
            _dbt.set(rubi);
        }
        retrieveComplexNQxB(_dbt);
    }
    finally {
        _dbt.close();
    }
}


The Schema for the PilotBeans looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<schema targetNamespace="testPeter" xmlns="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" attributeFormDefault="unqualified">
    <import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="../ext_schema/xml.xsd"/>

    <complexType name="PilotType">
        <attribute name="firstName" type="string" use="required"/>
        <attribute name="lastName" type="string" use="required"/>
        <attribute name="points" type="integer" use="required" />
    </complexType>
</schema>

The Pilot class is even simpler, as it doesn't distinguish between first and last name. But that should'nt have a big impact. The code for the PilotBeans is created with scomp from XmlBeans 2.2.0.


In both ways I end up with a database containing 2000 objects of Pilots or PilotBeans. But the database containing the PilotBeans stores several other objects that come with XmlBeans (probably the Schematype?) creating a massive overhead. The file sizes are:
TestSimplePilot.db4o    176 KB
TestXmlBeanPilot.db4o  5700 KB

This increase is unreasonable big. In the case for our actual project some 100 XML files with total size of only 1.5 MB created a 3 GB (!) db4o database file!!!

Does maybe anyone know a solution for this? I can only think of either dropping XmlBeans and using another parser / my own classes to represent and store the XML data in db4o. Or have a try with native XML databases. Has anyone had experiences with XmlBeans and databases?

Thanks for any replies,
cheers,
Peter
-- 
"Feel free" - 10 GB Mailbox, 100 FreeSMS/Monat ...
Jetzt GMX TopMail testen: www.gmx.net/de/go/mailfooter/topmail-out

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org


RE: XmlBeans and Databases (db4o)

Posted by Cezar Andrei <ce...@bea.com>.
There are different advantages and disadvantages for different saving approaches. When persistence is used there are different problems to be taken in consideration: storage space, performance - time to write and read, compatibility when schema is evolving, compatibility when application and libraries are evolving.

Depending on what is important and what is not for each application one can choose one or another approach.
 
I don't know db4o, but if it would have a way to configure the info that should save from XMLBeans you might get it to the same persistence size if compatibility is not an issue.

By default, an XMLBean is persisted to its XML representation which is big and slow, but it's the best bet for compatibility both for schema and application evolution. Binary XML tries to solve these problems, when it will become a standard we'll make use of it.

Cezar

> -----Original Message-----
> From: "Peter Schübel" [mailto:Peter.Schuebel@gmx.de]
> Sent: Monday, February 19, 2007 12:11 PM
> To: user@xmlbeans.apache.org
> Subject: XmlBeans and Databases (db4o)
> 
> Hello!
> 
> I am working on a project where various application data is initially read
> in from XML files. This data may later be updated through a remote
> connection. Via another remote connection, the data is made available to
> another computer that does further processing. Basically we have several
> sources that need to be managed. Therefore it is desirable to have a
> database storing and managing all this data. We decided to try the object-
> oriented database db4o (www.db4o.com). XML processing (as validation is a
> must!) is currently done with XMLBeans. It is very handy as we don't have
> to manually create the 66 classes for all of our 3 Schemas.
> 
> I searched the db4o forum and already found out that someone has had
> problems with XMLBeans. I also ran into a few, but managed to do some
> workarounds for them. Still there remains a fundamental problem that just
> makes the use of XMLBeans terribly inefficient in combination with db4o.
> I am talking about a (probably xmlbeans-induced) database file size
> increase of about 30 times compared to a simple example. I'd like to know
> if anyone can give me some advice on what best to do.
> 
> 
> Consider the following code, where i rebuilt the db4o Pilot example with
> XMLBeans (class PilotType, let's call them PilotBeans):
> 
> public static void main(String[] args) {
>     Db4o.configure().exceptionsOnNotStorable(true);
>     String databaseFilename = new
> String("Z:\\SVN\\tests\\db4o\\TestSimplePilot.db4o");
>     String databaseFilenameXB = new
> String("Z:\\SVN\\tests\\db4o\\TestXmlBeanPilot.db4o");
> 
>     ObjectContainer _db=Db4o.openFile(databaseFilename);
>     try {
>         clearDatabase(_db);
> 
>         System.out.print("Storing simple Objects...\n");
>         for(int i=0; i<1000; ++i) {
>             _db.set(new Pilot("Michael Schumacher",100));
>             _db.set(new Pilot("Rubens Barrichello",99));
>         }
>         retrieveComplexNQ(_db);
>     }
>     finally {
>         _db.close();
>     }
> 
>     ObjectContainer _dbt=Db4o.openFile(databaseFilenameXB);
>     try {
>         clearDatabaseXB(_dbt);
> 
>         System.out.print("Storing XmlBean Objects...\n");
>         for(int i=0; i<1000; ++i) {
>             PilotType michi = PilotType.Factory.newInstance();
>             PilotType rubi  = PilotType.Factory.newInstance();
>             michi.setFirstName("Michael");
>             michi.setLastName("Schumacher");
>             michi.setPoints(new BigInteger("100"));
>             rubi.setFirstName("Rubens");
>             rubi.setLastName("Barrichello");
>             rubi.setPoints(new BigInteger("99"));
> 
>             _dbt.set(michi);
>             _dbt.set(rubi);
>         }
>         retrieveComplexNQxB(_dbt);
>     }
>     finally {
>         _dbt.close();
>     }
> }
> 
> 
> The Schema for the PilotBeans looks like this:
> <?xml version="1.0" encoding="UTF-8"?>
> <schema targetNamespace="testPeter"
> xmlns="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"
> attributeFormDefault="unqualified">
>     <import namespace="http://www.w3.org/XML/1998/namespace"
> schemaLocation="../ext_schema/xml.xsd"/>
> 
>     <complexType name="PilotType">
>         <attribute name="firstName" type="string" use="required"/>
>         <attribute name="lastName" type="string" use="required"/>
>         <attribute name="points" type="integer" use="required" />
>     </complexType>
> </schema>
> 
> The Pilot class is even simpler, as it doesn't distinguish between first
> and last name. But that should'nt have a big impact. The code for the
> PilotBeans is created with scomp from XmlBeans 2.2.0.
> 
> 
> In both ways I end up with a database containing 2000 objects of Pilots or
> PilotBeans. But the database containing the PilotBeans stores several
> other objects that come with XmlBeans (probably the Schematype?) creating
> a massive overhead. The file sizes are:
> TestSimplePilot.db4o    176 KB
> TestXmlBeanPilot.db4o  5700 KB
> 
> This increase is unreasonable big. In the case for our actual project some
> 100 XML files with total size of only 1.5 MB created a 3 GB (!) db4o
> database file!!!
> 
> Does maybe anyone know a solution for this? I can only think of either
> dropping XmlBeans and using another parser / my own classes to represent
> and store the XML data in db4o. Or have a try with native XML databases.
> Has anyone had experiences with XmlBeans and databases?
> 
> Thanks for any replies,
> cheers,
> Peter
> --
> "Feel free" - 10 GB Mailbox, 100 FreeSMS/Monat ...
> Jetzt GMX TopMail testen: www.gmx.net/de/go/mailfooter/topmail-out
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
> For additional commands, e-mail: user-help@xmlbeans.apache.org

_______________________________________________________________________
Notice:  This email message, together with any attachments, may contain
information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated
entities,  that may be confidential,  proprietary,  copyrighted  and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: user-help@xmlbeans.apache.org