You are viewing a plain text version of this content. The canonical link for it is here.
Posted to j-dev@xerces.apache.org by Andy Wai/Toronto/IBM <aw...@ca.ibm.com> on 2000/11/22 19:02:48 UTC
Rampant object creation under Xerces J
We in the process of bring up a 4-tier application and XML over
MQSeries was chosen to transport things from the web server to
the application server and then from there to the backend integrator.
We originally used the IBM parser (for obvious reason...) and later
on migrated to Xerces-J as it was handed over to Apache.
Anyway, we're using the SAX parser within Xerces and things are
working as advertised. That is until we started stress testing
the system. We discovered that under stress, both the client and
the server are chewing up memory so fast that the garbage collector
just can't keep up.
We have eventually narrowed things down XML parsing. We created
multithreaded test clients and servers that transport messages via
a single MQSeries queue. With enough threads to generate 25 200 byte
messages per second, both client and server JVM have to have 128M
of memory to avoid momentary garbage collection freezes. Adding
more threads to bring throughput to 30 messages per second to
cause garbage collection freeze regardless of how much memory the
JVM is given.
Then I started looking at things using JProbe Below is a stand-
alone program that creates one SAX parser and send a very simple
XML document string through it 100 times sequentially. To simplify
things, I haven't even set any content handler. It does a forced
garbage collect just before it exits so JProbe can get an accurate
idea of what is garbage is what is not.
import org.xml.sax.*;
import org.xml.sax.helpers.*;
import java.io.*;
public class MemTest {
public static void main( String[] args ) {
TestReq testReq = new TestReq();
try {
XMLReader parser = XMLReaderFactory.createXMLReader(
"org.apache.xerces.parsers.SAXParser" );
parser.setFeature( "http://xml.org/sax/features/validation",
false );
String xmlDoc = "<?xml version=\"1.0\" standalone
=\"no\"?><TestReq><request>Testing 1, 2,3...</request></TestReq>";
for ( int i = 0; i < 100; ++i ) {
InputSource in = new InputSource(
new StringReader( xmlDoc ) );
System.out.println( "#" + i );
parser.parse( in );
}
Runtime.getRuntime().gc();
} catch (Exception e) {
e.printStackTrace();
}
}
}
With Xerces-J 1.2.0, JProbe reports total memory consumption of
around 14MB during the lifetime of the program. By the end of the
run, less than 300k of that is real allocated memory. That averages
to around 140k per message.
The garbage monitor in JProbe reports the highest garbage count
comes from utils.SymbolCache.addSymbolToCache() that created
18445 int[] objects of which only 355 were referenced at the end.
Java.util.Hashtable.put() created 8815 HashtableEntry object of
which 95 were referenced at the end. DatatypeValidatorFactoryImpl.
createDatatypeValidateor created 5601 Object[] of which none was
referenced at the end. Class.getConstructor also created 5601
Object[] of which none was referenced at the end. Then there are
a bunch of BigInteger related objects numbered into 3000+ which
are mostly free by the end as well.
At first, I tried to go into SymbolCache.java to find out what's
generating all the int[]. There are quite a few places, but the
main one is in addSymbolToCache() at line 199, which was called
17300 times. Then I started looking at the Hashtable.put and
discovered that of the 8912 calls to it, 2800 came from
DatatypeValidatorFactoryImpl.addValidator(), 4500 from
DataValidatorFactoryImpl.initializeRegistry(), 1600 from
DecimalDatatypeValidator.checkForFacetConsistency(), and 12
from ParseForXMLSchema.getRange(). And the BigInteger stuff is
similarly related to integer validation.
So looks like almost all of the garbage objects are validation
related. Then I check SymbolCache.addSymbolToCache() and found
that it's 94% called by StringPool.addSymbol() which is 78% called
by XMLValidator.init(). So it's validation related as well.
Since validation was explicitly turned off in the program above
and it's still generating all these objects, I was quite curious
as to what validation will do to memory usage. So I rerun the
test with validation on. And the numbers are almost identical
in both cases. Now the question to those in the know is, what's
the difference between validation and non-validation? I mean,
140k for non-validation seems pretty excessive. Are these objects
really necessary?
Given that the project is delivering at Dec 15, we need to have
this figured out real quick. What's my best course of action now?
Any insight would be greatly appreciated.
Andy Wai
e-Business Service
IBM Canada
Re: Rampant object creation under Xerces J
Posted by Eric Ye <er...@locus.apache.org>.
Can you also post how many objects were actually referenced when the
validation is on?
_____
Eric Ye * IBM, JTC - Silicon Valley * ericye@locus.apache.org
----- Original Message -----
From: "Andy Wai/Toronto/IBM" <aw...@ca.ibm.com>
To: <xe...@xml.apache.org>
Sent: Wednesday, November 22, 2000 10:02 AM
Subject: Rampant object creation under Xerces J
> We in the process of bring up a 4-tier application and XML over
> MQSeries was chosen to transport things from the web server to
> the application server and then from there to the backend integrator.
> We originally used the IBM parser (for obvious reason...) and later
> on migrated to Xerces-J as it was handed over to Apache.
>
> Anyway, we're using the SAX parser within Xerces and things are
> working as advertised. That is until we started stress testing
> the system. We discovered that under stress, both the client and
> the server are chewing up memory so fast that the garbage collector
> just can't keep up.
>
> We have eventually narrowed things down XML parsing. We created
> multithreaded test clients and servers that transport messages via
> a single MQSeries queue. With enough threads to generate 25 200 byte
> messages per second, both client and server JVM have to have 128M
> of memory to avoid momentary garbage collection freezes. Adding
> more threads to bring throughput to 30 messages per second to
> cause garbage collection freeze regardless of how much memory the
> JVM is given.
>
> Then I started looking at things using JProbe Below is a stand-
> alone program that creates one SAX parser and send a very simple
> XML document string through it 100 times sequentially. To simplify
> things, I haven't even set any content handler. It does a forced
> garbage collect just before it exits so JProbe can get an accurate
> idea of what is garbage is what is not.
>
> import org.xml.sax.*;
> import org.xml.sax.helpers.*;
> import java.io.*;
>
> public class MemTest {
> public static void main( String[] args ) {
> TestReq testReq = new TestReq();
> try {
> XMLReader parser = XMLReaderFactory.createXMLReader(
> "org.apache.xerces.parsers.SAXParser" );
> parser.setFeature( "http://xml.org/sax/features/validation",
> false );
> String xmlDoc = "<?xml version=\"1.0\" standalone
> =\"no\"?><TestReq><request>Testing 1, 2,3...</request></TestReq>";
> for ( int i = 0; i < 100; ++i ) {
> InputSource in = new InputSource(
> new StringReader( xmlDoc ) );
> System.out.println( "#" + i );
> parser.parse( in );
> }
> Runtime.getRuntime().gc();
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> }
>
> With Xerces-J 1.2.0, JProbe reports total memory consumption of
> around 14MB during the lifetime of the program. By the end of the
> run, less than 300k of that is real allocated memory. That averages
> to around 140k per message.
>
> The garbage monitor in JProbe reports the highest garbage count
> comes from utils.SymbolCache.addSymbolToCache() that created
> 18445 int[] objects of which only 355 were referenced at the end.
> Java.util.Hashtable.put() created 8815 HashtableEntry object of
> which 95 were referenced at the end. DatatypeValidatorFactoryImpl.
> createDatatypeValidateor created 5601 Object[] of which none was
> referenced at the end. Class.getConstructor also created 5601
> Object[] of which none was referenced at the end. Then there are
> a bunch of BigInteger related objects numbered into 3000+ which
> are mostly free by the end as well.
>
> At first, I tried to go into SymbolCache.java to find out what's
> generating all the int[]. There are quite a few places, but the
> main one is in addSymbolToCache() at line 199, which was called
> 17300 times. Then I started looking at the Hashtable.put and
> discovered that of the 8912 calls to it, 2800 came from
> DatatypeValidatorFactoryImpl.addValidator(), 4500 from
> DataValidatorFactoryImpl.initializeRegistry(), 1600 from
> DecimalDatatypeValidator.checkForFacetConsistency(), and 12
> from ParseForXMLSchema.getRange(). And the BigInteger stuff is
> similarly related to integer validation.
>
> So looks like almost all of the garbage objects are validation
> related. Then I check SymbolCache.addSymbolToCache() and found
> that it's 94% called by StringPool.addSymbol() which is 78% called
> by XMLValidator.init(). So it's validation related as well.
>
> Since validation was explicitly turned off in the program above
> and it's still generating all these objects, I was quite curious
> as to what validation will do to memory usage. So I rerun the
> test with validation on. And the numbers are almost identical
> in both cases. Now the question to those in the know is, what's
> the difference between validation and non-validation? I mean,
> 140k for non-validation seems pretty excessive. Are these objects
> really necessary?
>
> Given that the project is delivering at Dec 15, we need to have
> this figured out real quick. What's my best course of action now?
> Any insight would be greatly appreciated.
>
> Andy Wai
> e-Business Service
> IBM Canada
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-dev-help@xml.apache.org
>
>
RE: Rampant object creation under Xerces J
Posted by rsanford <rs...@nolimitsystems.com>.
whoah! that is a very impressive analysis.
i have to order jprobe.
rjsjr
> -----Original Message-----
> From: Andy Wai/Toronto/IBM [mailto:awai@ca.ibm.com]
> Sent: Wednesday, November 22, 2000 12:03 PM
> To: xerces-j-dev@xml.apache.org
> Subject: Rampant object creation under Xerces J
>
>
> We in the process of bring up a 4-tier application and XML over
> MQSeries was chosen to transport things from the web server to
> the application server and then from there to the backend integrator.
> We originally used the IBM parser (for obvious reason...) and later
> on migrated to Xerces-J as it was handed over to Apache.
>
> Anyway, we're using the SAX parser within Xerces and things are
> working as advertised. That is until we started stress testing
> the system. We discovered that under stress, both the client and
> the server are chewing up memory so fast that the garbage collector
> just can't keep up.
>
> We have eventually narrowed things down XML parsing. We created
> multithreaded test clients and servers that transport messages via
> a single MQSeries queue. With enough threads to generate 25 200 byte
> messages per second, both client and server JVM have to have 128M
> of memory to avoid momentary garbage collection freezes. Adding
> more threads to bring throughput to 30 messages per second to
> cause garbage collection freeze regardless of how much memory the
> JVM is given.
>
> Then I started looking at things using JProbe Below is a stand-
> alone program that creates one SAX parser and send a very simple
> XML document string through it 100 times sequentially. To simplify
> things, I haven't even set any content handler. It does a forced
> garbage collect just before it exits so JProbe can get an accurate
> idea of what is garbage is what is not.
>
> import org.xml.sax.*;
> import org.xml.sax.helpers.*;
> import java.io.*;
>
> public class MemTest {
> public static void main( String[] args ) {
> TestReq testReq = new TestReq();
> try {
> XMLReader parser = XMLReaderFactory.createXMLReader(
> "org.apache.xerces.parsers.SAXParser" );
> parser.setFeature( "http://xml.org/sax/features/validation",
> false );
> String xmlDoc = "<?xml version=\"1.0\" standalone
> =\"no\"?><TestReq><request>Testing 1, 2,3...</request></TestReq>";
> for ( int i = 0; i < 100; ++i ) {
> InputSource in = new InputSource(
> new StringReader( xmlDoc ) );
> System.out.println( "#" + i );
> parser.parse( in );
> }
> Runtime.getRuntime().gc();
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> }
>
> With Xerces-J 1.2.0, JProbe reports total memory consumption of
> around 14MB during the lifetime of the program. By the end of the
> run, less than 300k of that is real allocated memory. That averages
> to around 140k per message.
>
> The garbage monitor in JProbe reports the highest garbage count
> comes from utils.SymbolCache.addSymbolToCache() that created
> 18445 int[] objects of which only 355 were referenced at the end.
> Java.util.Hashtable.put() created 8815 HashtableEntry object of
> which 95 were referenced at the end. DatatypeValidatorFactoryImpl.
> createDatatypeValidateor created 5601 Object[] of which none was
> referenced at the end. Class.getConstructor also created 5601
> Object[] of which none was referenced at the end. Then there are
> a bunch of BigInteger related objects numbered into 3000+ which
> are mostly free by the end as well.
>
> At first, I tried to go into SymbolCache.java to find out what's
> generating all the int[]. There are quite a few places, but the
> main one is in addSymbolToCache() at line 199, which was called
> 17300 times. Then I started looking at the Hashtable.put and
> discovered that of the 8912 calls to it, 2800 came from
> DatatypeValidatorFactoryImpl.addValidator(), 4500 from
> DataValidatorFactoryImpl.initializeRegistry(), 1600 from
> DecimalDatatypeValidator.checkForFacetConsistency(), and 12
> from ParseForXMLSchema.getRange(). And the BigInteger stuff is
> similarly related to integer validation.
>
> So looks like almost all of the garbage objects are validation
> related. Then I check SymbolCache.addSymbolToCache() and found
> that it's 94% called by StringPool.addSymbol() which is 78% called
> by XMLValidator.init(). So it's validation related as well.
>
> Since validation was explicitly turned off in the program above
> and it's still generating all these objects, I was quite curious
> as to what validation will do to memory usage. So I rerun the
> test with validation on. And the numbers are almost identical
> in both cases. Now the question to those in the know is, what's
> the difference between validation and non-validation? I mean,
> 140k for non-validation seems pretty excessive. Are these objects
> really necessary?
>
> Given that the project is delivering at Dec 15, we need to have
> this figured out real quick. What's my best course of action now?
> Any insight would be greatly appreciated.
>
> Andy Wai
> e-Business Service
> IBM Canada
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-dev-help@xml.apache.org
>
RE: Rampant object creation under Xerces J
Posted by rsanford <rs...@nolimitsystems.com>.
question - are you creating an instance of the parser for every
message or are you reusing parser instances?
the system that i have in place is handling 100,000 documents
per hour with each document on the order of 20k of which 15k
is contained in a base64 encoded cdata section.
i have a pool of message handlers each of which contains its
own parser instance that is instantiated when the handler is
instantiated and remains for the lifetime of the handler
which is essentially until the server is rebooted. this
reduces the amount of memory that is reallocated over time.
would this help solve your problem at all?
rjsjr
> -----Original Message-----
> From: Andy Wai/Toronto/IBM [mailto:awai@ca.ibm.com]
> Sent: Wednesday, November 22, 2000 12:03 PM
> To: xerces-j-dev@xml.apache.org
> Subject: Rampant object creation under Xerces J
>
>
> We in the process of bring up a 4-tier application and XML over
> MQSeries was chosen to transport things from the web server to
> the application server and then from there to the backend integrator.
> We originally used the IBM parser (for obvious reason...) and later
> on migrated to Xerces-J as it was handed over to Apache.
>
> Anyway, we're using the SAX parser within Xerces and things are
> working as advertised. That is until we started stress testing
> the system. We discovered that under stress, both the client and
> the server are chewing up memory so fast that the garbage collector
> just can't keep up.
>
> We have eventually narrowed things down XML parsing. We created
> multithreaded test clients and servers that transport messages via
> a single MQSeries queue. With enough threads to generate 25 200 byte
> messages per second, both client and server JVM have to have 128M
> of memory to avoid momentary garbage collection freezes. Adding
> more threads to bring throughput to 30 messages per second to
> cause garbage collection freeze regardless of how much memory the
> JVM is given.
>
> Then I started looking at things using JProbe Below is a stand-
> alone program that creates one SAX parser and send a very simple
> XML document string through it 100 times sequentially. To simplify
> things, I haven't even set any content handler. It does a forced
> garbage collect just before it exits so JProbe can get an accurate
> idea of what is garbage is what is not.
>
> import org.xml.sax.*;
> import org.xml.sax.helpers.*;
> import java.io.*;
>
> public class MemTest {
> public static void main( String[] args ) {
> TestReq testReq = new TestReq();
> try {
> XMLReader parser = XMLReaderFactory.createXMLReader(
> "org.apache.xerces.parsers.SAXParser" );
> parser.setFeature( "http://xml.org/sax/features/validation",
> false );
> String xmlDoc = "<?xml version=\"1.0\" standalone
> =\"no\"?><TestReq><request>Testing 1, 2,3...</request></TestReq>";
> for ( int i = 0; i < 100; ++i ) {
> InputSource in = new InputSource(
> new StringReader( xmlDoc ) );
> System.out.println( "#" + i );
> parser.parse( in );
> }
> Runtime.getRuntime().gc();
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> }
>
> With Xerces-J 1.2.0, JProbe reports total memory consumption of
> around 14MB during the lifetime of the program. By the end of the
> run, less than 300k of that is real allocated memory. That averages
> to around 140k per message.
>
> The garbage monitor in JProbe reports the highest garbage count
> comes from utils.SymbolCache.addSymbolToCache() that created
> 18445 int[] objects of which only 355 were referenced at the end.
> Java.util.Hashtable.put() created 8815 HashtableEntry object of
> which 95 were referenced at the end. DatatypeValidatorFactoryImpl.
> createDatatypeValidateor created 5601 Object[] of which none was
> referenced at the end. Class.getConstructor also created 5601
> Object[] of which none was referenced at the end. Then there are
> a bunch of BigInteger related objects numbered into 3000+ which
> are mostly free by the end as well.
>
> At first, I tried to go into SymbolCache.java to find out what's
> generating all the int[]. There are quite a few places, but the
> main one is in addSymbolToCache() at line 199, which was called
> 17300 times. Then I started looking at the Hashtable.put and
> discovered that of the 8912 calls to it, 2800 came from
> DatatypeValidatorFactoryImpl.addValidator(), 4500 from
> DataValidatorFactoryImpl.initializeRegistry(), 1600 from
> DecimalDatatypeValidator.checkForFacetConsistency(), and 12
> from ParseForXMLSchema.getRange(). And the BigInteger stuff is
> similarly related to integer validation.
>
> So looks like almost all of the garbage objects are validation
> related. Then I check SymbolCache.addSymbolToCache() and found
> that it's 94% called by StringPool.addSymbol() which is 78% called
> by XMLValidator.init(). So it's validation related as well.
>
> Since validation was explicitly turned off in the program above
> and it's still generating all these objects, I was quite curious
> as to what validation will do to memory usage. So I rerun the
> test with validation on. And the numbers are almost identical
> in both cases. Now the question to those in the know is, what's
> the difference between validation and non-validation? I mean,
> 140k for non-validation seems pretty excessive. Are these objects
> really necessary?
>
> Given that the project is delivering at Dec 15, we need to have
> this figured out real quick. What's my best course of action now?
> Any insight would be greatly appreciated.
>
> Andy Wai
> e-Business Service
> IBM Canada
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-dev-help@xml.apache.org
>