You are viewing a plain text version of this content. The canonical link for it is here.
Posted to j-dev@xerces.apache.org by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org> on 2012/07/14 04:51:33 UTC
[jira] [Created] (XERCESJ-1573) Possible memory leak when using
assertions
Jorge L. Williams created XERCESJ-1573:
------------------------------------------
Summary: Possible memory leak when using assertions
Key: XERCESJ-1573
URL: https://issues.apache.org/jira/browse/XERCESJ-1573
Project: Xerces2-J
Issue Type: Bug
Components: XML Schema 1.1 Datatypes
Affects Versions: 2.11.0
Reporter: Jorge L. Williams
Priority: Blocker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414371#comment-13414371 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
I would agree that, using assertions is more resource consuming than lot of other XSD constructs.
The Xerces implementation, builds a full XDM tree and maintains a stack to traverse nested assertions. Assertions also requires instantiating the XPath 2.0 engine, and asking it to evaluate assertions. The advise to save memory during XSD validation is, to use assertions only when you need them and declare them down the XML tree as much as possible (so that the assert XDM tree footprint is small).
In your specific <assert> example (i.e, xsd:dateTime(@end) le (xsd:dateTime(@start) + xsd:dayTimeDuration('P1D'))), the <assert> XPath 2.0 engine may be following an expensive evaluation path (I haven't analyzed this case really, but as a general rule certain XPath expressions may result in poor runtime performance).
I'm not sure just now, how we can approach to make the implementation better for your bug report. Also not sure whether it's right to classify this issue as Bug->blocker. May be a less severe classification of this is more appropriate (improvement etc).
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, duration-sans-assert.xsd, duration.xsd, good.xml
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414743#comment-13414743 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
Hi Jorge,
After your latest report, I did analyzed the java heap dump while running the jaxp.SourceValidator tool with your example validation having <assert> and with 1000000 iterations.
It seems, that the way class PSVIElementNSImpl is managed in XMLAssertPsychopathXPath2Impl.java is causing lot of objects of PSVIElementNSImpl to be alive during validation.
In the following code fragment of XMLAssertPsychopathXPath2Impl.java,
public void startElement(QName element, XMLAttributes attributes, Augmentations augs) throws Exception {
if (fCurrentAssertDomNode == null) {
fCurrentAssertDomNode = new PSVIElementNSImpl((CoreDocumentImpl) fAssertDocument, element.uri, element.rawname);
fAssertDocument.appendChild(fCurrentAssertDomNode);
}
...
}
the object creation with the call, "new PSVIElementNSImpl(...)" is perhaps the pain point for memory usage. Your report https://issues.apache.org/jira/secure/attachment/12536525/mem.png also points to a similar analysis.
It seems, that this malady is felt only when you stress the validator to a point like 1000000 iterations. A repetition like 100, 1000 or 10000 works nicely for me.
Here are some of my final thoughts related to this bug report,
1) It's just too risky to change the logic of call startElement(...) of XMLAssertPsychopathXPath2Impl.java (of the areas of code I've pointed above) -- if this is really the pain point after my quick analysis. These parts of Xerces <assert> validator are the foundation of our assertions implementation. It may be doable to optimize these code paths, but that would require careful thought.
2) Perhaps a usage of <assert> is not suitable for a stress situation like you've mentioned. Or you may perhaps restructure your validation program, to take out assertions to another layer of your application.
3) This may be a issue of using <assert> via the jaxp.SourceValidator tool. If you can implement an application reset at a user layer, instead of expecting Xerces to do optimizations for this case, that might also help.
Thanks,
Mukul
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414759#comment-13414759 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
I'm sorry Muku, but I do think that this is a bug in Xerces, let me explain.
First late me state that this is not an optimization issue -- this is a critical bug in Xerces.
Second, the issue is not an issue with jaxp.SourceValidator -- I'm only using the jaxp.SourceValidator to illustrate it. We've run into the issue in our own app.
The issue is with your implementation of JAX-P.
JAX-P allows a validator (or a validator handler) to be continuously reused..
This is feature is used quite frequently server side when validating messages.
In a JAX-P validator: http://docs.oracle.com/javase/6/docs/api/javax/xml/validation/Validator.html#reset%28%29
In a JAX-P validator handler startDocument should also allow for a full reset. According to http://docs.oracle.com/javase/6/docs/api/javax/xml/validation/ValidatorHandler.html
"A ValidatorHandler is automatically reset every time the startDocument method is invoked."
The problem, it seems, is that you are not correctly resetting the Psychopath implementation after call to reset() .. or startDocument is encountered so that memory continues to accumulate.
Consider that when you have a server side application, a call to validate a message against an XML schema may be made millions of times a day. This is a usecase that xerces is handling today. I would argue that it's a use case that you should continue to support.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414533#comment-13414533 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
Looks like it happens with these checks as well add images (d3-d6) to show that the same pattern holds. Also added mem.png that shows a sample of memory while the code was executing -- most of the classes there come from Xerces.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Updated] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jorge L. Williams updated XERCESJ-1573:
---------------------------------------
Attachment: mem.png
d3.png
d6.png
d5.png
d4.png
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Comment Edited] (XERCESJ-1573) Possible memory leak when
using assertions
Posted by "Michael Glavassevich (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417375#comment-13417375 ]
Michael Glavassevich edited comment on XERCESJ-1573 at 7/18/12 7:12 PM:
------------------------------------------------------------------------
Mukul, as a member of the JSR 206 expert group which developed the JAXP 1.4 spec I can assure you that Validators are intended for reuse and is the best practice we recommend for users.
We have a memory leak. We need to fix it. If you're not sure how to approach it, I can take an initial stab at a fix.
was (Author: mrglavas@ca.ibm.com):
Mukul, as a member of the JSR 206 expert group which developed JAXP 1.4 spec I can assure you that Validators are intended for reuse and is the best practice we recommend for users. We have a memory leak. We need to fix it.
If you're not sure how to approach it, I can take an initial stab at a fix.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415921#comment-13415921 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
@ Michael : thanks for the analysis. When I move the following calls,
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
before the for loop, I can see the same leaks as reported by Jorge earlier (i.e the instances of PSVIElementNSImpl are getting accumulated in
XMLAssertPsychopathXPath2Impl).
We'll try, if we can improve memory usage within XMLAssertPsychopathXPath2Impl.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414787#comment-13414787 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
I agree Michael. I'll try to look at these parts of code base, for better memory management.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415667#comment-13415667 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
I've just attached a test case (named XS11Validate.java) to iterate XSD 1.1 validations. When I do validation using this test file, using good.xml & duration.xsd posted by Jorge iterating 1000000 times I take a heap dump of the running JVM after about a minute of execution and I get following memory usage results (i.e the leak suspects),
org.eclipse.wst.xml.xpath2.processor.internal.XPathFlex : 196,520 (20.92%) bytes
org.eclipse.wst.xml.xpath2.processor.internal.XPathCup : 95,696 (10.19%) bytes
This time, the heap dump tool doesn't report that the object PSVIElementNSImpl is a leak suspect and the overall memory consumption is reasonable. This makes me believe, that the way jaxp.SourceValidator tool is running repeating validations is consuming more memory then for e.g, this test case.
I therefore, cannot accept the argument that Xerces is using memory inefficiently while using XSD 1.1 assertions!
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Updated] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mukul Gandhi updated XERCESJ-1573:
----------------------------------
Attachment: XS11Validate.java
adding a test case, to iterate xsd 1.1 validations
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Updated] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jorge L. Williams updated XERCESJ-1573:
---------------------------------------
Attachment: d2.png
d1.png
good.xml
duration.xsd
duration-sans-assert.xsd
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, duration-sans-assert.xsd, duration.xsd, good.xml
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414726#comment-13414726 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
Awesome. Thanks.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Michael Glavassevich (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415778#comment-13415778 ]
Michael Glavassevich commented on XERCESJ-1573:
-----------------------------------------------
I don't believe it either, but can explain why Mukul didn't see the problem.
A new Validator is being created on each loop in XS11Validate.java:
for (int indx = 0; indx < repeat; indx++) {
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
validator.validate(new StreamSource(xmlDoc));
}
Validators are reusable objects. For good performance it is best practice to reuse them and expect users of Xerces to be doing that.
Move the creation of the Validator outside of the loop you'll see the leak.
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
for (int indx = 0; indx < repeat; indx++) {
validator.validate(new StreamSource(xmlDoc));
}
XMLAssertPsychopathXPath2Impl keeps appending elements to the same Document.
It would like this if you serialized it, where the number of 'test' elements == number of iterations. So yes, this is definitely leaking and not being reinitialized properly.
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z">
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
...
</test>
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414764#comment-13414764 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
Took a quick glance at the code. I don't think you need to make a change to startElement at all. I think that all that you have to do is propagate the startDocument element all the way to XMLAssertPsychopathXPath2Impl and set fAssertDocument = new PSVIDocumentImpl() at that point. You might want to reinitialize other variables as well.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Updated] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jorge L. Williams updated XERCESJ-1573:
---------------------------------------
Attachment: d8.png
d7.png
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, d7.png, d8.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417359#comment-13417359 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
@ Jorge: as per following documentation about the JAXP Validator, http://xerces.apache.org/xerces2-j/javadocs/api/javax/xml/validation/Validator.html
"A validator is a thread-unsafe and non-reentrant object."
Therefore I would not expect ideally the Validator object to be reused after a schema validation assessment.
Also, the workaround I suggested earlier in this thread (i.e, to create a new Validator object for each iteration of the loop -- which naturally resets the Validator object's memory), seem to solve the memory bottleneck problem.
Please correct me, if I'm wrong.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Comment Edited] (XERCESJ-1573) Possible memory leak when
using assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414439#comment-13414439 ]
Jorge L. Williams edited comment on XERCESJ-1573 at 7/14/12 4:25 PM:
---------------------------------------------------------------------
Hi Mukul,
I totally understand that there's an overhead associated with assertions. I want to point out some things though.
1. The instance document is really small:
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
building the XDM tree should not be an expensive operation in terms of memory or CPU.
2. If you look at the graph (d2.png) you'll note that for the first couple of runs we're able to validate the instance doc with less than 10 Megs of memory (similar to the non-assert case). As the runs progresses memory usage starts increasing dramatically finally peaking at 120MB -- before I stopped the run (couldn't get it to finish because it was going so slow). Performing garbage collection did not improve the memory situation -- this tells me that there may be a leak of some kind.
3. The behavior is more dramatic when you include more than one assertion in the XSD. We are doing server side validation and memory usage quickly grows to gigabytes of memory -- until the app eventually grinds to a halt. So in that sense the bug *is* a complete blocker for us -- we can't use Xerces to validate those XSDs.
4. The Saxon implementation validates the same XSD with minimal decreases in performance or increases in memory usage.
5. I did a simple experiment where I, removed the assertion from the XSD, created a DOM for the instance document, performed the XPath query (using Saxon), then passed the doc to Xerces2 for validation I saw no issues in terms of performance or memory usage.
was (Author: jorgew):
Hi Mukul,
I totally understand that there's an overhead associated with assertions. I want to point out some things though.
1. The instance document is really small:
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
building the XDM tree should not be an expensive operation in terms of memory or CPU.
2. If you look at the graph (d2.png) you'll note that for the first couple of runs we're able to validate the instance doc with less than 10 Megs of memory (similar to the non-assert case). As the runs progresses memory usage starts increasing dramatically finally peaking at 120MB -- before I stopped the run (couldn't get it to finish because it was going so slow). Performing garbage collection did not improve the memory situation -- this tells me that there may be a leak of some kind.
3. The behavior is more dramatic when you include more than one assertion in the XSD. We are doing server side validation and memory usage quickly grows to gigabytes of memory -- until the app eventually grinds to a halt. So in that sense the bug *is* a complete blocker for us -- we can't use Xerces to validate those XSDs.
4. The Saxon implementation validates the same XSD with minimal increases in performance and memory usage.
5. I did a simple experiment where I, removed the assertion from the XSD, created a DOM for the instance document, performed the XPath query (using Saxon), then passed the doc to Xerces2 for validation I saw no issues in terms of performance or memory usage.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, duration-sans-assert.xsd, duration.xsd, good.xml
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414273#comment-13414273 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
Been playing around with the latest 1.1 branch and I think I'm seeing a memory leak when an XSD uses assertions.
I'm including the following files:
duration.xsd : A simple XSD with 1.1 assertions
duration-sans-assert.xsd: Same XSD without assertions
good.xml : a valid instance document according to both XSDs
I'm also attaching screen shots of memory usage in jconsole when running SourceValidator to validate the instance document 1,000,000 times. As the job was running I periodically pressed the "Perform GC" button.
With duration-sans-assert.xsd memory usage always returns back to about ~4MB. With duration.xsd memory usage just keeps going up.
The performance differences between the two is not insignificant
I've noticed the same behavior in my own app.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, duration-sans-assert.xsd, duration.xsd, good.xml
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414780#comment-13414780 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
if you think that this would help, can you please create a working patch and post. If this modification doesn't fail any of the existing things, we can apply the patch to code base.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414698#comment-13414698 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
Are you sure that you're reusing the XPath engine correctly between iterations?
[mukul] I'm really not sure. But I'll try to find out.
Anyway, thanks for reporting these issues. We'll try to analyze memory leak issues at our end, and will try to improve the implementation.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Michael Glavassevich (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418284#comment-13418284 ]
Michael Glavassevich commented on XERCESJ-1573:
-----------------------------------------------
@Mukul: I agree that your change resolves the leak, and it's good that it's fixed but we can do better than that. Recreating components each time the validator is used isn't in the spirit of Xerces' design. We're spending cycles creating the same objects over and over again, churning the garbage collector. Perhaps you could resolve this JIRA issue now (since the leak is fixed), but I think this really should be revisited.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, d7.png, d8.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414491#comment-13414491 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
My guess is, that every usage of XSD 1.1 assertions may not result in same performance problem that you've reported. Perhaps the performance problem that you've reported, is related to specific class of XPath expressions (for e.g, that build date objects or do date arithmetic) -- perhaps you can help us identify these things.
As a second check, you may do for e.g same performance tests with following XSD type definitions,
<!-- tests if a number is even -->
<xs:simpleType name="evenNumber">
<xs:restriction base="xs:integer">
<xs:assert test="$value mod 2 = 0"/>
</xs:restriction>
</xs:simpleType>
<!-- enforce cardinality of a sequence via an assertion -->
<xs:complexType>
<xs:sequence>
<xs:element name="a" type="xs:string" maxOccurs="unbounded"/>
</xs:sequence>
<xs:assert test="count(a) = 5"/>
</xs:complexType>
and report to us the results.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, duration-sans-assert.xsd, duration.xsd, good.xml
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Comment Edited] (XERCESJ-1573) Possible memory leak when
using assertions
Posted by "Michael Glavassevich (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415778#comment-13415778 ]
Michael Glavassevich edited comment on XERCESJ-1573 at 7/16/12 11:30 PM:
-------------------------------------------------------------------------
I don't believe it either, but can explain why Mukul didn't see the problem.
A new Validator is being created on each loop in XS11Validate.java:
for (int indx = 0; indx < repeat; indx++) {
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
validator.validate(new StreamSource(xmlDoc));
}
Validators are reusable objects. For good performance it is best practice to reuse them and expect users of Xerces to be doing that.
Move the creation of the Validator outside of the loop and you'll see the leak.
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
for (int indx = 0; indx < repeat; indx++) {
validator.validate(new StreamSource(xmlDoc));
}
XMLAssertPsychopathXPath2Impl keeps appending elements to the same Document.
It would look like this if you serialized it, where the number of 'test' elements == number of iterations. So yes, this is definitely leaking and not being reinitialized properly.
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z">
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
...
</test>
was (Author: mrglavas@ca.ibm.com):
I don't believe it either, but can explain why Mukul didn't see the problem.
A new Validator is being created on each loop in XS11Validate.java:
for (int indx = 0; indx < repeat; indx++) {
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
validator.validate(new StreamSource(xmlDoc));
}
Validators are reusable objects. For good performance it is best practice to reuse them and expect users of Xerces to be doing that.
Move the creation of the Validator outside of the loop you'll see the leak.
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
for (int indx = 0; indx < repeat; indx++) {
validator.validate(new StreamSource(xmlDoc));
}
XMLAssertPsychopathXPath2Impl keeps appending elements to the same Document.
It would look like this if you serialized it, where the number of 'test' elements == number of iterations. So yes, this is definitely leaking and not being reinitialized properly.
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z">
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
...
</test>
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Comment Edited] (XERCESJ-1573) Possible memory leak when
using assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414764#comment-13414764 ]
Jorge L. Williams edited comment on XERCESJ-1573 at 7/15/12 8:20 PM:
---------------------------------------------------------------------
Took a quick glance at the code. I don't think you need to make a change to startElement at all. I think that all that you have to do is propagate the startDocument event all the way to XMLAssertPsychopathXPath2Impl and set fAssertDocument = new PSVIDocumentImpl() at that point. You might want to reinitialize other variables as well.
was (Author: jorgew):
Took a quick glance at the code. I don't think you need to make a change to startElement at all. I think that all that you have to do is propagate the startDocument element all the way to XMLAssertPsychopathXPath2Impl and set fAssertDocument = new PSVIDocumentImpl() at that point. You might want to reinitialize other variables as well.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Resolved] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mukul Gandhi resolved XERCESJ-1573.
-----------------------------------
Resolution: Fixed
Assignee: Mukul Gandhi
resolving this issue, since the fix for this has been committed to SVN.
@ Michael : we'll try to do a detailed reset of the assertions validator sometime soon in future. many thanks for your useful analysis, for this issue.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Assignee: Mukul Gandhi
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, d7.png, d8.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414599#comment-13414599 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
Mukul,
The initial set of validations are successfully occurring with < 20 MB of memory. So if I can validate the document with 20 MB, why would validating the document again cost more memory? It shouldn't, the number of repetitions should not matter at all --- unless you're carrying over data from one iteration to the next instead of freeing it -- in which case it's a leak.
After running for a long time on our app, we literally start consuming GBs of data causing it to grind to a halt.
If this issue is not fixed, then I don't see how anyone can use Xerces in a server environment.
Are you sure that you're reusing the XPath engine correctly between iterations?
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414439#comment-13414439 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
Hi Mukul,
I totally understand that there's an overhead associated with assertions. I want to point out some things though.
1. The instance document is really small:
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
building the XDM tree should not be an expensive operation in terms of memory or CPU.
2. If you look at the graph (d2.png) you'll note that for the first couple of runs we're able to validate the instance doc with less than 10 Megs of memory (similar to the non-assert case). As the runs progresses memory usage starts increasing dramatically finally peaking at 120MB -- before I stopped the run (couldn't get it to finish because it was going so slow). Performing garbage collection did not improve the memory situation -- this tells me that there may be a leak of some kind.
3. The behavior is more dramatic when you include more than one assertion in the XSD. We are doing server side validation and memory usage quickly grows to gigabytes of memory -- until the app eventually grinds to a halt. So in that sense the bug *is* a complete blocker for us -- we can't use Xerces to validate those XSDs.
4. The Saxon implementation validates the same XSD with minimal increases in performance and memory usage.
5. I did a simple experiment where I, removed the assertion from the XSD, created a DOM for the instance document, performed the XPath query (using Saxon), then passed the doc to Xerces2 for validation I saw no issues in terms of performance or memory usage.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, duration-sans-assert.xsd, duration.xsd, good.xml
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Comment Edited] (XERCESJ-1573) Possible memory leak when
using assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414273#comment-13414273 ]
Jorge L. Williams edited comment on XERCESJ-1573 at 7/14/12 2:55 AM:
---------------------------------------------------------------------
Been playing around with the latest 1.1 branch and I think I'm seeing a memory leak when an XSD uses assertions.
I'm including the following files:
duration.xsd : A simple XSD with 1.1 assertions
duration-sans-assert.xsd: Same XSD without assertions
good.xml : a valid instance document according to both XSDs
I'm also attaching screen shots of memory usage in jconsole when running SourceValidator to validate the instance document 1,000,000 times. As the job was running I periodically pressed the "Perform GC" button.
With duration-sans-assert.xsd memory usage always returns back to about ~7-8MB. With duration.xsd memory usage just keeps going up till it reaches somewhere about 120MB.
The performance differences between the two is not insignificant
I've noticed the same behavior in my own app.
was (Author: jorgew):
Been playing around with the latest 1.1 branch and I think I'm seeing a memory leak when an XSD uses assertions.
I'm including the following files:
duration.xsd : A simple XSD with 1.1 assertions
duration-sans-assert.xsd: Same XSD without assertions
good.xml : a valid instance document according to both XSDs
I'm also attaching screen shots of memory usage in jconsole when running SourceValidator to validate the instance document 1,000,000 times. As the job was running I periodically pressed the "Perform GC" button.
With duration-sans-assert.xsd memory usage always returns back to about ~4MB. With duration.xsd memory usage just keeps going up.
The performance differences between the two is not insignificant
I've noticed the same behavior in my own app.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, duration-sans-assert.xsd, duration.xsd, good.xml
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417768#comment-13417768 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
That certainly solves the memory problem. Looks like you're recreating the XPath engine on startDocument? I'm wondering instead if you should be resetting the engine -- if possible.
The performance difference is huge when you add assertions.
no assertions: good.xml: 78801/1000000=0.078801 ms
assertions: good.xml: 1171421/1000000=1.171421 ms
It took a little less than 2 mins to complete a million iterations on duration-sans-assert.xsd (d7.png)
close to 20 mins on duration.xsd (d8.png).
I'm not seeing that sort of performance degradation with the saxon impl. Not sure if that's related to this fix though -- I'll pop up a profiler to it when I get a chance.
Thanks
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Michael Glavassevich (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414781#comment-13414781 ]
Michael Glavassevich commented on XERCESJ-1573:
-----------------------------------------------
I haven't looked too deeply at this but certainly see the potential for leaks in XMLAssertPsychopathXPath2Impl. It doesn't seem to have an explicit reset() back to its initial state. Aside from the DOM processing which looks suspect, there are stacks where items are pushed, but may never be popped if an exception occurs in the middle of processing. Without a call to Stack.clear() when they're reused they may just keep growing (i.e. leaking memory) until the JVM eventually runs out of memory.
Other components in Xerces were designed to reinitialize themselves on reuse. Looks like this one needs a revisit.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415768#comment-13415768 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
I don't believe it :-)
Can you attach a jconsole graph? Can you run it for more than 1 min?
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Michael Glavassevich (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417375#comment-13417375 ]
Michael Glavassevich commented on XERCESJ-1573:
-----------------------------------------------
Mukul, as a member of the JSR 206 expert group which developed JAXP 1.4 spec I can assure you that Validators are intended for reuse and is the best practice we recommend for users. We have a memory leak. We need to fix it.
If you're not sure how to approach it, I can take an initial stab at a fix.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417678#comment-13417678 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
@ Michael : I've just committed a fix for this (SVN revision 1363110). The new code base, now doesn't result in a memory leak as before.
@ Jorge : you may verify, if the fix solves the problem.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418262#comment-13418262 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
@ Jorge : yes, now the assertions validator is initialized in startDocument() call and reset in endDocument(). this makes the algorithm to optimize assertions memory usage quite simpler. I'm not sure just now, if we should go for a detailed reset of the assertions validator -- for now, I would like to go with the current fix.
as per your report, with the latest fix one validation assessment having an assertion taking 1.171421 ms to run looks good to me. I can also see memory consumption to be quite better this time than before, for your example.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, d7.png, d8.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Mukul Gandhi (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414574#comment-13414574 ]
Mukul Gandhi commented on XERCESJ-1573:
---------------------------------------
thanks for sharing the reports. I can see, that for all the examples you've tried for memory profiling, the peak memory usage is about 125 Mb. But this result is, when you've repeated the validation assessment 10000000 times. I'm not sure, if this a poor memory management by Xerces (since the repetitions were quite large).
The Xerces XSD 1.1 assertions implementation, uses DOM objects to implement the XPath 2.0 XDM model (which is a given, due to internal design of PsychoPath XPath engine) -- which I guess, leads to the implementation using more memory.
I see no easy way, of improving further the implementation quickly by design to optimize memory management. Unfortunately, my response to this issue would be "won't fix" due to the risks involved.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Comment Edited] (XERCESJ-1573) Possible memory leak when
using assertions
Posted by "Michael Glavassevich (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415778#comment-13415778 ]
Michael Glavassevich edited comment on XERCESJ-1573 at 7/16/12 11:29 PM:
-------------------------------------------------------------------------
I don't believe it either, but can explain why Mukul didn't see the problem.
A new Validator is being created on each loop in XS11Validate.java:
for (int indx = 0; indx < repeat; indx++) {
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
validator.validate(new StreamSource(xmlDoc));
}
Validators are reusable objects. For good performance it is best practice to reuse them and expect users of Xerces to be doing that.
Move the creation of the Validator outside of the loop you'll see the leak.
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
for (int indx = 0; indx < repeat; indx++) {
validator.validate(new StreamSource(xmlDoc));
}
XMLAssertPsychopathXPath2Impl keeps appending elements to the same Document.
It would look like this if you serialized it, where the number of 'test' elements == number of iterations. So yes, this is definitely leaking and not being reinitialized properly.
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z">
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
...
</test>
was (Author: mrglavas@ca.ibm.com):
I don't believe it either, but can explain why Mukul didn't see the problem.
A new Validator is being created on each loop in XS11Validate.java:
for (int indx = 0; indx < repeat; indx++) {
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
validator.validate(new StreamSource(xmlDoc));
}
Validators are reusable objects. For good performance it is best practice to reuse them and expect users of Xerces to be doing that.
Move the creation of the Validator outside of the loop you'll see the leak.
Validator validator = schema.newValidator();
validator.setErrorHandler(this);
for (int indx = 0; indx < repeat; indx++) {
validator.validate(new StreamSource(xmlDoc));
}
XMLAssertPsychopathXPath2Impl keeps appending elements to the same Document.
It would like this if you serialized it, where the number of 'test' elements == number of iterations. So yes, this is definitely leaking and not being reinitialized properly.
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z">
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
<test xmlns="http://www.rackspace.com/test/simple"
start="2012-03-12T11:51:11Z" end="2012-03-13T11:51:11Z"/>
...
</test>
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org
[jira] [Commented] (XERCESJ-1573) Possible memory leak when using
assertions
Posted by "Jorge L. Williams (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESJ-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417371#comment-13417371 ]
Jorge L. Williams commented on XERCESJ-1573:
--------------------------------------------
While it's true that a Validator is not thread safe, that doesn't mean that you can't reuse one within a single thread of execution. In fact, Validators are designed to be re-used as Michael stated in a previous email.
from the JavaDocs above -- in the definition of "reset"
"Validator is reset to the same state as when it was created with Schema.newValidator(). reset() is designed to allow the reuse of existing Validators thus saving resources associated with the creation of new Validators."
I would also note that there are no thread safety issues with the example of the validator being initialized outside of the loop -- since there is only a single thread of execution.
> Possible memory leak when using assertions
> ------------------------------------------
>
> Key: XERCESJ-1573
> URL: https://issues.apache.org/jira/browse/XERCESJ-1573
> Project: Xerces2-J
> Issue Type: Bug
> Components: XML Schema 1.1 Datatypes
> Affects Versions: 2.11.0
> Reporter: Jorge L. Williams
> Priority: Blocker
> Attachments: XS11Validate.java, d1.png, d2.png, d3.png, d4.png, d5.png, d6.png, duration-sans-assert.xsd, duration.xsd, good.xml, mem.png
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org