You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ant.apache.org by "Stephen Haberman (JIRA)" <ji...@apache.org> on 2012/10/25 23:47:12 UTC
[jira] [Created] (IVY-1386) Disable DTD external fetching when not
validation
Stephen Haberman created IVY-1386:
-------------------------------------
Summary: Disable DTD external fetching when not validation
Key: IVY-1386
URL: https://issues.apache.org/jira/browse/IVY-1386
Project: Ivy
Issue Type: Bug
Affects Versions: trunk
Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
Reporter: Stephen Haberman
Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IVY-1386) Disable DTD external fetching when not
validation
Posted by "Stephen Haberman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Haberman updated IVY-1386:
----------------------------------
Attachment: ivyde-xml-dtd-hung.txt
Stack trace of IvyDE hung on XMLHelper.parse.
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVY-1386
> URL: https://issues.apache.org/jira/browse/IVY-1386
> Project: Ivy
> Issue Type: Bug
> Affects Versions: trunk
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Moved] (IVYDE-329) Disable DTD external fetching when not
validation
Posted by "Maarten Coene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVYDE-329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maarten Coene moved IVY-1386 to IVYDE-329:
------------------------------------------
Affects Version/s: (was: trunk)
Issue Type: Improvement (was: Bug)
Key: IVYDE-329 (was: IVY-1386)
Project: IvyDE (was: Ivy)
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVYDE-329
> URL: https://issues.apache.org/jira/browse/IVYDE-329
> Project: IvyDE
> Issue Type: Improvement
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IVY-1386) Disable DTD external fetching when not
validation
Posted by "Stephen Haberman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Haberman updated IVY-1386:
----------------------------------
Attachment: disableExternalDtds.diff
Fix to disable external DTD fetching.
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVY-1386
> URL: https://issues.apache.org/jira/browse/IVY-1386
> Project: Ivy
> Issue Type: Bug
> Affects Versions: trunk
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVYDE-329) Disable DTD external fetching when
not validation
Posted by "Maarten Coene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVYDE-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497392#comment-13497392 ]
Maarten Coene commented on IVYDE-329:
-------------------------------------
I've moved this issue to the IvyDE project because I think if we fix the issue it should be done in the IvyDE codebase.
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVYDE-329
> URL: https://issues.apache.org/jira/browse/IVYDE-329
> Project: IvyDE
> Issue Type: Improvement
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVY-1386) Disable DTD external fetching when
not validation
Posted by "Stephen Haberman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484534#comment-13484534 ]
Stephen Haberman commented on IVY-1386:
---------------------------------------
Hm...good point; I will try that and let you know.
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVY-1386
> URL: https://issues.apache.org/jira/browse/IVY-1386
> Project: Ivy
> Issue Type: Bug
> Affects Versions: trunk
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVY-1386) Disable DTD external fetching when
not validation
Posted by "Maarten Coene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485490#comment-13485490 ]
Maarten Coene commented on IVY-1386:
------------------------------------
I also did a quick test
for instance, consider this xml:
{code:xml}
<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<!DOCTYPE doc SYSTEM \"Test.dtd\">
<doc>a &foo;</doc>
{code}
and this Test.dtd:
{code}
<!ENTITY foo "foo">
{code}
Without the patch, the SAX handler receives 'a foo' as content of the <doc>-element.
When I apply your patch, the SAX handler receives 'a' as content. So it seems the &foo; entity is ignored.
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVY-1386
> URL: https://issues.apache.org/jira/browse/IVY-1386
> Project: Ivy
> Issue Type: Bug
> Affects Versions: trunk
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVY-1386) Disable DTD external fetching when
not validation
Posted by "Maarten Coene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484531#comment-13484531 ]
Maarten Coene commented on IVY-1386:
------------------------------------
I'm wondering what would happen if the XML file contains entities defined in an external DTD when you disable external DTD fetching?
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVY-1386
> URL: https://issues.apache.org/jira/browse/IVY-1386
> Project: Ivy
> Issue Type: Bug
> Affects Versions: trunk
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVY-1386) Disable DTD external fetching when
not validation
Posted by "Stephen Haberman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484590#comment-13484590 ]
Stephen Haberman commented on IVY-1386:
---------------------------------------
I ran a test program against an XML file with entities in it, both valid and invalid entities, and the updated XMLHelper with external-dtd-fetching=false doesn't seem to affect anything.
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVY-1386
> URL: https://issues.apache.org/jira/browse/IVY-1386
> Project: Ivy
> Issue Type: Bug
> Affects Versions: trunk
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVYDE-329) Disable DTD external fetching when
not validation
Posted by "Leon Ephorus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVYDE-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500978#comment-13500978 ]
Leon Ephorus commented on IVYDE-329:
------------------------------------
We also encountered this problem. I rolled back to the following (released) versions for which the problems do not occur:
{noformat}
Apache Ivy 2.3.0.cr2_20121105223351
Apache Ivy Ant Tasks 2.3.0.cr2_20121105223351
Apache IvyDE 2.2.0.beta1-201203282058-RELEASE
Apache IvyDE Resolve Visualizer 2.2.0.beta1-201203282058-RELEASE
{noformat}
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVYDE-329
> URL: https://issues.apache.org/jira/browse/IVYDE-329
> Project: IvyDE
> Issue Type: Improvement
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVY-1386) Disable DTD external fetching when
not validation
Posted by "Stephen Haberman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485495#comment-13485495 ]
Stephen Haberman commented on IVY-1386:
---------------------------------------
Hi Maarten, thanks for taking the time to test my patch--I apologize for not finding this out before asking you take time to look at it.
I have been running a patched version of Ivy with my proposed fix in my local Eclipse, and thanks so far work, so I was lulled into thinking the semantics where the same. Obviously it does not, so I'll look some more in to this.
What do you think of adding another arg to XMLHelper.parse, called loadExternalDtds, which defaults=true, but can be set false?
The reason is that IvyDE's IvySettingsContentDescriptor, AFAICT, does not need external DTDs to work, because it uses a custom default handler to just look for the root element of "ivysettings" and then stops.
Given how often Eclipse asks IvyDE this question (once per XML file in the project, on a regular basis, AFAICT), it seems useful to allow IvySettingsContentDescriptor to opt out of DTD loading.
I can work up a patch that does this, if you approve. Of if you have other suggestions, let me know and I can try those out too.
Thanks!
> Disable DTD external fetching when not validation
> -------------------------------------------------
>
> Key: IVY-1386
> URL: https://issues.apache.org/jira/browse/IVY-1386
> Project: Ivy
> Issue Type: Bug
> Affects Versions: trunk
> Environment: Ivy 2.4.0.alpha/IvyDE 2.2.0.beta2
> Reporter: Stephen Haberman
> Attachments: disableExternalDtds.diff, ivyde-xml-dtd-hung.txt
>
>
> Our team uses Ivy/IvyDE, and noticed Eclipse hanging today/yesterday, to the point of being unusable.
> I tracked it down to Eclipse asking IvyDE "is this your file?", which IvyDE's IvySettingsContentDescriber used XMLHelper.parse to answer, but then Xerces hung while trying to load the XML file's DTD.
> So, the problem was the DTD being unavailable, but it seems like XMLHelper should turn this off, especially if the schema parameter is null, and it's in non-validating mode.
> The attached patch turns off external DTD fetching when XMLHelper is already in non-validating mode.
> This avoids the wire call, which speeds up the XMLHelper.parse by at least 100%, ~250-300ms when fetching the DTD, to ~100-150ms when not. (And this is the happy case, in the worst case, waiting for the timeout if the DTD is unavailable, takes ~20s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira