Performance and thread safety in Xerces

[Dennis Thrysoe - Netnord A/S <dt...@netnord.dk>]

Hi,

I have a few questions that I hope somebody can help me shed some light on:

Is the DOM implementation in Xerces 1.0.3 inefficient in some places?

I my concrete case I have some code that calls ElementImpl.getAttribute()
about 320,000 times. This code takes a bit long, and creates a total of
almost 600,000 objects.

Is this a known problem? If it is, is it handlede in newer releases?

The other thing relates to the thread safety of Xerces. I have two seperate
threads parsing two different XML files. Still I seem to get race conditions
on some arrays:

For instance this situation:

(Read/Write Race): Thread1
  read from
<unknown array>
  at
boolean org.apache.xerces.validators.dtd.EntityPool.ensureCapacity(int):160
  and
thread2
  wrote at
void org.apache.xerces.utils.ChunkyCharArray.append(char):88

In my case I get a total of 8 race conditions.

I hope I'm helping, and that I can get helped :)

-dennis

Re: Performance and thread safety in Xerces

[Arnaud Le Hors <le...@us.ibm.com>]

Dennis Thrysoe - Netnord A/S wrote:
> 
> Is the DOM implementation in Xerces 1.0.3 inefficient in some places?

Yes.

> 
> I my concrete case I have some code that calls ElementImpl.getAttribute()
> about 320,000 times. This code takes a bit long, and creates a total of
> almost 600,000 objects.
> 
> Is this a known problem? If it is, is it handlede in newer releases?

I've made a lot of improvements with regards to memory foot print in our
DOM implementation since 1.0.3. So try a newer version and you should
see a real difference! Feedback on this is very welcome actually. I
haven't got any (which I guess is good...)
-- 
Arnaud  Le Hors - IBM Cupertino, XML Technology Group

Re: Performance and thread safety in Xerces

[Mike Pogue <mp...@apache.org>]

Hmmmm.  The section on thread safety seems to have disappeared from the Xerces-J docs.
There is a section in Xerces-C, however: 
http://xml.apache.org/xerces-c/faq-parse.html#faq-5

The bottom line:

	1) Making the parser 100% thread safe would be a HUGE, GIGANTIC performance hit.
		(Did I say LARGE, BIG, MAJOR performance hit?  That, too.)
	2) Your application knows best what needs to be locked, and what doesn't.
	3) Therefore, although you can run the parser, DOM, etc. from separate threads,
		we do NOT lock every method call on the parser, and each application
		is responsible for doing appropriate locking.

Mike

Dennis Thrysoe - Netnord A/S wrote:
> 
> > Dennis Thrysoe - Netnord A/S wrote:
> > > I my concrete case I have some code that calls
> > ElementImpl.getAttribute()
> > > about 320,000 times. This code takes a bit long, and creates a total of
> > > almost 600,000 objects.
> >
> > I would imagine that your 600,000 objects come from the fact that
> > the getAttribute method has to concatenate the children of the
> > attribute node in order to return you a string. So save the
> > memory you should use getAttributeNode and then query the
> > children yourself.
> 
> Using AttrImpl.getValue() has the same effect: Creating many objects. Which
> otrher way is there to query the attributes value?
> 
> > > The other thing relates to the thread safety of Xerces. I have
> > two seperate
> > > threads parsing two different XML files. Still I seem to get
> > race conditions
> > > on some arrays:
> >
> > Access to the parser and the DOM are *not* thread-safe. You still
> > have to manage locks in order to ensure that multiple threads are
> >
> >   1) not using the parser at the same time
> >   2) not editing a DOM document at the same time
> >
> > Now, if there are static members that are used in the code and
> > access to them is not synchronized, then that could be causing
> > the race condition problems that you are seeing. Let us know if
> > you deduce the cause of your problem.
> 
> Does plans exist for making the parser thread safe? In our project it is
> quite important that several threads can each parse something different, or
> the whole idea of concurrency is wasted.
> 
> -dennis
> 
> ---------------------------------------------------------------------
> In case of troubles, e-mail:     webmaster@xml.apache.org
> To unsubscribe, e-mail:          general-unsubscribe@xml.apache.org
> For additional commands, e-mail: general-help@xml.apache.org

RE: Performance and thread safety in Xerces

[Dennis Thrysoe - Netnord A/S <dt...@netnord.dk>]

> Dennis Thrysoe - Netnord A/S wrote:
> > I my concrete case I have some code that calls
> ElementImpl.getAttribute()
> > about 320,000 times. This code takes a bit long, and creates a total of
> > almost 600,000 objects.
>
> I would imagine that your 600,000 objects come from the fact that
> the getAttribute method has to concatenate the children of the
> attribute node in order to return you a string. So save the
> memory you should use getAttributeNode and then query the
> children yourself.


Using AttrImpl.getValue() has the same effect: Creating many objects. Which
otrher way is there to query the attributes value?


> > The other thing relates to the thread safety of Xerces. I have
> two seperate
> > threads parsing two different XML files. Still I seem to get
> race conditions
> > on some arrays:
>
> Access to the parser and the DOM are *not* thread-safe. You still
> have to manage locks in order to ensure that multiple threads are
>
>   1) not using the parser at the same time
>   2) not editing a DOM document at the same time
>
> Now, if there are static members that are used in the code and
> access to them is not synchronized, then that could be causing
> the race condition problems that you are seeing. Let us know if
> you deduce the cause of your problem.


Does plans exist for making the parser thread safe? In our project it is
quite important that several threads can each parse something different, or
the whole idea of concurrency is wasted.

-dennis

Re: Performance and thread safety in Xerces

[Andy Clark <an...@apache.org>]

Dennis Thrysoe - Netnord A/S wrote:
> I my concrete case I have some code that calls ElementImpl.getAttribute()
> about 320,000 times. This code takes a bit long, and creates a total of
> almost 600,000 objects.

I would imagine that your 600,000 objects come from the fact that
the getAttribute method has to concatenate the children of the
attribute node in order to return you a string. So save the 
memory you should use getAttributeNode and then query the
children yourself.

> The other thing relates to the thread safety of Xerces. I have two seperate
> threads parsing two different XML files. Still I seem to get race conditions
> on some arrays:

Access to the parser and the DOM are *not* thread-safe. You still
have to manage locks in order to ensure that multiple threads are

  1) not using the parser at the same time
  2) not editing a DOM document at the same time

Now, if there are static members that are used in the code and
access to them is not synchronized, then that could be causing
the race condition problems that you are seeing. Let us know if
you deduce the cause of your problem.

-- 
Andy Clark * IBM, JTC - Silicon Valley * andyc@apache.org