You are viewing a plain text version of this content. The canonical link for it is here.
Posted to j-users@xerces.apache.org by sa...@ca.ibm.com on 2002/06/20 01:44:50 UTC
Re: xerces-j 2 sample ASBuilder - AS performance
Did you call parseASXXX to parse the schema(s), and call setASModel to set
them back onto the ASBuilder? You need to do that to reuse the preparsed
schema(s).
Sandy Gao
Software Developer, IBM Canada
(1-905) 413-3255
sandygao@ca.ibm.com
Shengyou Zeng
<szeng@cisco.c To: xerces-j-user@xml.apache.org
om> cc:
Subject: xerces-j 2 sample ASBuilder - AS performance
06/07/2002
07:15 PM
Please respond
to
xerces-j-user
The sample program dom.ASBuilder shipped with Xerces-j 2.0.1
"illustrates how to preparse xml schema documents and how to
validate instance documents against preparsed schema grammars."
(quotes from DOM Sample page for xerces-j 2). I was hoping that,
because of schema preparsing, the performance of ASBuilder
should improve compared to straight DOM 2. However, that is not
the case. As a matter of fact, the DOM 3 implementation with
schema preparsing fares worse than DOM 2 xerces-j
(org.apache.xerces.parsers.DOMParser).
For example, in my case, using the same schema and the same xml
instance (parsing and validating 100x), ASBuilder costs 10000 ms while
DOMParser costs 8000 ms. (In case you are wondering, I did set
schema validation and namespace features to true for the DOMParser).
My understanding is that Abstract Schema preparses schema and validation
of repeat xml documents uses the preparsed schema. This is what sample
ASBuilder is supposed to demonstrate. Thus using Abstract Schema should
save schema (loading and) parsing time. But why don't see the performance
gain one would expect? What am I missing?
Thanks.
-Shengyou
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
RE: OutOfMemoryError using Xerces 1.4.3
Posted by Rob Outar <ro...@ideorlando.org>.
Does anyone know the ratio of xml file to DOM Object? Does Xerces 2 handle
memory better than Xerces 1?
Thanks,
Rob
-----Original Message-----
From: Jan Dvorak [mailto:jan.dvorak@mathan.cz]
Sent: Friday, June 21, 2002 12:19 PM
To: xerces-j-user@xml.apache.org; Rob Outar
Subject: Re: OutOfMemoryError using Xerces 1.4.3
Hello Rob,
A DOM tree for a 25 meg XML file may take hundreds of megs in memory.
You can increase the JVM memory limit with the -Xmx option.
Jan
Rob Outar wrote:
> Hello all,
>
> I am parsing a 25 meg XML file using Xerces 1.4.3 DOMParser and I keep
> getting an OutOfMemory Exception. I know this process will take some
time,
> but I did not expect to be out of memory. Is this normal for this size
> file? Can Xerces 2 or some other parser better handle memory?
>
> Let me know,
>
> Rob
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
Failed to create document object of class: org.apache.xerces.dom.DeferredDocumentImpl
Posted by Rob Outar <ro...@ideorlando.org>.
I am setting this property:
parser.setProperty
("http://apache.org/xml/properties/dom/document-class-name",
"org.apache.xerces.dom.DeferredDocumentImpl");
but when I parse my xml file I get the below error:
Failed to create document object of class:
org.apache.xerces.dom.DeferredDocumentImpl
Any help would be appreciated.
Thanks,
Rob
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
Re: OutOfMemoryError using Xerces 1.4.3
Posted by Jan Dvorak <ja...@mathan.cz>.
None I'm aware of, but then, I don't regularly work with DOM.
I've seen estimates that the DOM can take 5-10 times the size of the XML file.
XSLT processors like Xalan or Saxon use their own lightweight DOM
implementations that are tuned to the application. The main difference is
that their DOM's don't support the tree modification operations. If you are
brave, you could do the same.
Perhaps you don't really need the whole thing in memory at once.
Dom4j, an alternative to W3C DOM under Java (www.dom4j.org), has a great
feature of ElementHandler. As you parse your XML, you can build only portions
of the tree in memory, work with XPath over these portions, and throw the
portions when you're done with them. E.g., a data batch can have a header
(which may be a portion you want to keep for the whole time of your
processing) and zillions of data records (which you will process one by one
anyway), so you throw away the record when you've processed it.
Jan
Rob Outar wrote:
| Is there any way to lower the amount of memory Xerces uses to build the DOM
| tree? A setting, feature, property? If so, what impact will it have on
| the DOM?
|
| Thanks,
|
| Rob
|
| -----Original Message-----
| From: Jan Dvorak [mailto:jan.dvorak@mathan.cz]
| Sent: Friday, June 21, 2002 12:19 PM
| To: xerces-j-user@xml.apache.org; Rob Outar
| Subject: Re: OutOfMemoryError using Xerces 1.4.3
|
|
| Hello Rob,
|
| A DOM tree for a 25 meg XML file may take hundreds of megs in memory.
| You can increase the JVM memory limit with the -Xmx option.
|
| Jan
|
| Rob Outar wrote:
| > Hello all,
| >
| > I am parsing a 25 meg XML file using Xerces 1.4.3 DOMParser and I keep
| > getting an OutOfMemory Exception. I know this process will take some
|
| time,
|
| > but I did not expect to be out of memory. Is this normal for this size
| > file? Can Xerces 2 or some other parser better handle memory?
| >
| > Let me know,
| >
| > Rob
| >
| >
| > ---------------------------------------------------------------------
| > To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
| > For additional commands, e-mail: xerces-j-user-help@xml.apache.org
|
| ---------------------------------------------------------------------
| To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
| For additional commands, e-mail: xerces-j-user-help@xml.apache.org
|
|
| ---------------------------------------------------------------------
| To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
| For additional commands, e-mail: xerces-j-user-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
RE: OutOfMemoryError using Xerces 1.4.3
Posted by Rob Outar <ro...@ideorlando.org>.
Is there any way to lower the amount of memory Xerces uses to build the DOM
tree? A setting, feature, property? If so, what impact will it have on the
DOM?
Thanks,
Rob
-----Original Message-----
From: Jan Dvorak [mailto:jan.dvorak@mathan.cz]
Sent: Friday, June 21, 2002 12:19 PM
To: xerces-j-user@xml.apache.org; Rob Outar
Subject: Re: OutOfMemoryError using Xerces 1.4.3
Hello Rob,
A DOM tree for a 25 meg XML file may take hundreds of megs in memory.
You can increase the JVM memory limit with the -Xmx option.
Jan
Rob Outar wrote:
> Hello all,
>
> I am parsing a 25 meg XML file using Xerces 1.4.3 DOMParser and I keep
> getting an OutOfMemory Exception. I know this process will take some
time,
> but I did not expect to be out of memory. Is this normal for this size
> file? Can Xerces 2 or some other parser better handle memory?
>
> Let me know,
>
> Rob
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
Re: OutOfMemoryError using Xerces 1.4.3
Posted by Jan Dvorak <ja...@mathan.cz>.
Hello Rob,
A DOM tree for a 25 meg XML file may take hundreds of megs in memory.
You can increase the JVM memory limit with the -Xmx option.
Jan
Rob Outar wrote:
> Hello all,
>
> I am parsing a 25 meg XML file using Xerces 1.4.3 DOMParser and I keep
> getting an OutOfMemory Exception. I know this process will take some time,
> but I did not expect to be out of memory. Is this normal for this size
> file? Can Xerces 2 or some other parser better handle memory?
>
> Let me know,
>
> Rob
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
OutOfMemoryError using Xerces 1.4.3
Posted by Rob Outar <ro...@ideorlando.org>.
Hello all,
I am parsing a 25 meg XML file using Xerces 1.4.3 DOMParser and I keep
getting an OutOfMemory Exception. I know this process will take some time,
but I did not expect to be out of memory. Is this normal for this size
file? Can Xerces 2 or some other parser better handle memory?
Let me know,
Rob
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org
Re: xerces-j 2 sample ASBuilder - AS performance
Posted by Shengyou Zeng <sz...@cisco.com>.
Hi Sandy,
The answers to both of your questions are "yes". As a matter
of fact, I simply made a copy of the ASBuilder.java file in the sample, added
timestamp statement in the appropriate places to measure
parsing costs, and then used the program to test a schema that
I defined and an xml instance document conformant to the schema.
Here is the code snippet (you can see it is basically the same as
the ASBuilder.java in the samples):
//
// PARSING XML SCHEMAS
//
try
{
ASModel asmodel = null;
for (i = 0; i < asfiles.size(); i++)
{
asmodel = parser.parseASURI((String)asfiles.elementAt(i));
parser.setAbstractSchema(asmodel);
}
}
catch (Exception e)
{
e.printStackTrace();
System.exit(1);
}
parser.setEntityResolver(EntityCache.instance());
try
{
long a = System.currentTimeMillis();
for (int j=0; j<100; j++)
parser.parseURI("http://localhost:8004/pcsp/test.xml");
long b = System.currentTimeMillis() - a;
System.out.println("total time = "+b);
System.exit(1);
}
catch (Exception e){}
I passed the schema file as one of the argument to main(). The exact code
in the sample builds an AbstractModel and sets it on the parser. Then the
parser is used to parse an xml instance document 100 times. I expect that
the subsequent rounds of parsing of this document use the same preparsed
schema and thus there should be an performance improvement over non-AS
parser, such as a DOMParser. But that was not what I saw - AS seems to
provide no performance gain.
-Shengyou
At 07:44 PM 06/19/2002 -0400, sandygao@ca.ibm.com wrote:
>Did you call parseASXXX to parse the schema(s), and call setASModel to set
>them back onto the ASBuilder? You need to do that to reuse the preparsed
>schema(s).
>
>Sandy Gao
>Software Developer, IBM Canada
>(1-905) 413-3255
>sandygao@ca.ibm.com
>
>
>
>
>
> Shengyou
> Zeng
>
> <szeng@cisco.c To:
> xerces-j-user@xml.apache.org
> om> cc:
>
> Subject: xerces-j 2 sample
> ASBuilder - AS performance
> 06/07/2002
>
> 07:15
> PM
>
> Please
> respond
>
> to
>
> xerces-j-user
>
>
>
>
>
>
>
>
>
>The sample program dom.ASBuilder shipped with Xerces-j 2.0.1
>"illustrates how to preparse xml schema documents and how to
> validate instance documents against preparsed schema grammars."
>(quotes from DOM Sample page for xerces-j 2). I was hoping that,
>because of schema preparsing, the performance of ASBuilder
>should improve compared to straight DOM 2. However, that is not
>the case. As a matter of fact, the DOM 3 implementation with
>schema preparsing fares worse than DOM 2 xerces-j
>(org.apache.xerces.parsers.DOMParser).
>For example, in my case, using the same schema and the same xml
>instance (parsing and validating 100x), ASBuilder costs 10000 ms while
>DOMParser costs 8000 ms. (In case you are wondering, I did set
>schema validation and namespace features to true for the DOMParser).
>
>My understanding is that Abstract Schema preparses schema and validation
>of repeat xml documents uses the preparsed schema. This is what sample
>ASBuilder is supposed to demonstrate. Thus using Abstract Schema should
>save schema (loading and) parsing time. But why don't see the performance
>gain one would expect? What am I missing?
>
>Thanks.
>
>-Shengyou
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
>For additional commands, e-mail: xerces-j-user-help@xml.apache.org
>
>
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
>For additional commands, e-mail: xerces-j-user-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org