You are viewing a plain text version of this content. The canonical link for it is here.

Posted to xindice-users@xml.apache.org by va...@edb4tel.com on 2002/04/02 10:41:15 UTC

Importing large numbers of XML files to Xindice

Hi all,

My goal is to fill an Xindice database with 500.000 docs and then try to run
certain test on this database.
After importing around 360.000 docs successfully, Xindice started giving the
error below for each xml-document to import. 
The xml-files are all identical. Before starting the import job I generated 3
indexes, one of them is unique, the other two represent large numbers of
documents having the same value on a certain xml-tag.
Does anybody know what this error is about?

Thanks in advance! 

PS.
I use the command 'xindice add_multiple_documents'.
The operating system is HP-UX.

-----------------------------------------
The error:
-----------------------------------------
Error Adding File: 0136148.xml
DEBUG>
java.io.EOFException
        at java.io.DataInputStream.readInt(Unknown Source)
        at
org.apache.xindice.xml.sax.SAXEventGenerator.processContainer(SAXEventGenerat
or.java:254)
        at
org.apache.xindice.xml.sax.SAXEventGenerator.start(SAXEventGenerator.java:342
)
        at
org.apache.xindice.core.indexer.IndexManager$SAXHandler.<init>(IndexManager.j
ava:462)
        at
org.apache.xindice.core.indexer.IndexManager$SAXHandler.<init>(IndexManager.j
ava:478)
        at
org.apache.xindice.core.indexer.IndexManager.removeDocument(IndexManager.java
:405)
        at
org.apache.xindice.core.Collection.putDocument(Collection.java:626)
        at
org.apache.xindice.core.Collection.insertDocument(Collection.java:659)
        at
org.apache.xindice.client.corba.CollectionServant.insertDocument(CollectionSe
rvant.java:317)
        at
org.apache.xindice.client.corba.db.CollectionPOA._invoke(CollectionPOA.java:1
81)
        at org.openorb.adapter.poa.POA.dispatch(POA.java:975)
        at
org.openorb.net.AbstractServerRequest.dispatch(AbstractServerRequest.java:750
)
        at
org.openorb.net.ServerManagerImpl.serve_request(ServerManagerImpl.java:1467)
        at
org.openorb.net.ServerManagerImpl.thread_pool_main(ServerManagerImpl.java:141
0)
        at
org.openorb.net.ServerManagerImpl.access$200(ServerManagerImpl.java:77)
        at
org.openorb.net.ServerManagerImpl$PoolThread.run(ServerManagerImpl.java:1557)
DEBUG>
org.apache.xindice.core.filer.FilerException: Error performing flush!
        at org.apache.xindice.core.filer.Paged.flush(Paged.java:421)
        at
org.apache.xindice.core.filer.BTreeFiler.flush(BTreeFiler.java:223)
        at
org.apache.xindice.core.filer.BTreeFiler.writeRecord(BTreeFiler.java:175)
        at
org.apache.xindice.core.Collection.putDocument(Collection.java:629)
        at
org.apache.xindice.core.Collection.insertDocument(Collection.java:659)
        at
org.apache.xindice.client.corba.CollectionServant.insertDocument(CollectionSe
rvant.java:317)
        at
org.apache.xindice.client.corba.db.CollectionPOA._invoke(CollectionPOA.java:1
81)
        at org.openorb.adapter.poa.POA.dispatch(POA.java:975)
        at
org.openorb.net.AbstractServerRequest.dispatch(AbstractServerRequest.java:750
)
        at
org.openorb.net.ServerManagerImpl.serve_request(ServerManagerImpl.java:1467)
        at
org.openorb.net.ServerManagerImpl.thread_pool_main(ServerManagerImpl.java:141
0)
        at
org.openorb.net.ServerManagerImpl.access$200(ServerManagerImpl.java:77)
        at
org.openorb.net.ServerManagerImpl$PoolThread.run(ServerManagerImpl.java:1557)
ERROR : Critical Error

Re: Importing large numbers of XML files to Xindice

Posted by Heinrich Götzger <go...@gmx.net>.

Vassilios,

just a question beside:
How many documents per second are you able to store?

Thanks.

Reards,

Heinrich
--
http://www.xmlBlaster.org

On Tue, 2 Apr 2002 vassilios.kalantzakos@edb4tel.com wrote:

>Hi all,
>
>My goal is to fill an Xindice database with 500.000 docs and then try to run
>certain test on this database.
>After importing around 360.000 docs successfully, Xindice started giving the
>error below for each xml-document to import.
>The xml-files are all identical. Before starting the import job I generated 3
>indexes, one of them is unique, the other two represent large numbers of
>documents having the same value on a certain xml-tag.
>Does anybody know what this error is about?
>
>Thanks in advance!
>
>PS.
>I use the command 'xindice add_multiple_documents'.
>The operating system is HP-UX.
>
>-----------------------------------------
>The error:
>-----------------------------------------
>Error Adding File: 0136148.xml
>DEBUG>
>java.io.EOFException
>        at java.io.DataInputStream.readInt(Unknown Source)
>        at
>org.apache.xindice.xml.sax.SAXEventGenerator.processContainer(SAXEventGenerat
>or.java:254)
>        at
>org.apache.xindice.xml.sax.SAXEventGenerator.start(SAXEventGenerator.java:342
>)
>        at
>org.apache.xindice.core.indexer.IndexManager$SAXHandler.<init>(IndexManager.j
>ava:462)
>        at
>org.apache.xindice.core.indexer.IndexManager$SAXHandler.<init>(IndexManager.j
>ava:478)
>        at
>org.apache.xindice.core.indexer.IndexManager.removeDocument(IndexManager.java
>:405)
>        at
>org.apache.xindice.core.Collection.putDocument(Collection.java:626)
>        at
>org.apache.xindice.core.Collection.insertDocument(Collection.java:659)
>        at
>org.apache.xindice.client.corba.CollectionServant.insertDocument(CollectionSe
>rvant.java:317)
>        at
>org.apache.xindice.client.corba.db.CollectionPOA._invoke(CollectionPOA.java:1
>81)
>        at org.openorb.adapter.poa.POA.dispatch(POA.java:975)
>        at
>org.openorb.net.AbstractServerRequest.dispatch(AbstractServerRequest.java:750
>)
>        at
>org.openorb.net.ServerManagerImpl.serve_request(ServerManagerImpl.java:1467)
>        at
>org.openorb.net.ServerManagerImpl.thread_pool_main(ServerManagerImpl.java:141
>0)
>        at
>org.openorb.net.ServerManagerImpl.access$200(ServerManagerImpl.java:77)
>        at
>org.openorb.net.ServerManagerImpl$PoolThread.run(ServerManagerImpl.java:1557)
>DEBUG>
>org.apache.xindice.core.filer.FilerException: Error performing flush!
>        at org.apache.xindice.core.filer.Paged.flush(Paged.java:421)
>        at
>org.apache.xindice.core.filer.BTreeFiler.flush(BTreeFiler.java:223)
>        at
>org.apache.xindice.core.filer.BTreeFiler.writeRecord(BTreeFiler.java:175)
>        at
>org.apache.xindice.core.Collection.putDocument(Collection.java:629)
>        at
>org.apache.xindice.core.Collection.insertDocument(Collection.java:659)
>        at
>org.apache.xindice.client.corba.CollectionServant.insertDocument(CollectionSe
>rvant.java:317)
>        at
>org.apache.xindice.client.corba.db.CollectionPOA._invoke(CollectionPOA.java:1
>81)
>        at org.openorb.adapter.poa.POA.dispatch(POA.java:975)
>        at
>org.openorb.net.AbstractServerRequest.dispatch(AbstractServerRequest.java:750
>)
>        at
>org.openorb.net.ServerManagerImpl.serve_request(ServerManagerImpl.java:1467)
>        at
>org.openorb.net.ServerManagerImpl.thread_pool_main(ServerManagerImpl.java:141
>0)
>        at
>org.openorb.net.ServerManagerImpl.access$200(ServerManagerImpl.java:77)
>        at
>org.openorb.net.ServerManagerImpl$PoolThread.run(ServerManagerImpl.java:1557)
>ERROR : Critical Error
>