You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jaxme-dev@ws.apache.org by "Robert Eric Reeves (JIRA)" <ja...@ws.apache.org> on 2007/08/30 00:12:32 UTC
[jira] Created: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
startElement method larger than 64KB when using a large amount of complexType's
-------------------------------------------------------------------------------
Key: JAXME-94
URL: https://issues.apache.org/jira/browse/JAXME-94
Project: JaxMe
Issue Type: Improvement
Components: JaxMe Core
Affects Versions: 0.5, 0.6
Environment: All
Reporter: Robert Eric Reeves
Priority: Critical
Fix For: 0.5, 0.6
Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Commented: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Jochen Wiedmann (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524019 ]
Jochen Wiedmann commented on JAXME-94:
--------------------------------------
Had some time yesterday to inspect it. My first impressions:
- This won't make it into the maintenance branch. Although the test suite
runs fine, I would still consider this as a disruptive change. Such changes
are reserved for the unstable branch.
- I have a couple of performance issues. Some of them are basically nitpicks,
but at least two of them are serious:
* The regular expression pattern can be generated by the generator. It
doesn't need to be generated at runtime.
* The map and the compiled pattern can be static and created when the class
is loaded.
Question to Jose: Do you prefer to fix this for yourself or do you want me to change this?
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Assignee: Jochen Wiedmann
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Commented: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Robert Eric Reeves (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530516 ]
Robert Eric Reeves commented on JAXME-94:
-----------------------------------------
Jochen,
Has Jose's patches passed muster to be included in the trunk?
FWIW, we're using his 0.52 patch with JDK 1.4 and its working like a champ and passes the unit tests.
Thanks!
Robert
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Assignee: Jochen Wiedmann
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: Jaxme94-SolutionB-0.5.2.diff, Jaxme94-SolutionB-0.5.3-SNAPSHOT.diff, Jaxme94-SolutionB-0.6-SNAPSHOT.diff, test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff, veryLongSequences-refactor-patch-0.5.2.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Commented: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Jose Luis Huertas (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525234 ]
Jose Luis Huertas commented on JAXME-94:
----------------------------------------
I send you my first approach to this issue that was not disruptive at all, so I think it can be applied to the maintenance branch.
It doesn't reduce the total amount of generated code, just refactors it dividing into private methods (one for each state), but it works.
I have included an unit test based on the example provided by Robert (with 100 elements) that fails before applying the patch (the generated classes don't compile) and pass after applying it.
With 250 elements the solution still works, but the generated code is huge and probably you will need to change the ant heap settings to not run out of memory.
Regarding to the previous patch, I'm trying to change the method Matcher.hitEnd() for one equivalent that works with JDK 1.4, but I still have not found a suitable one :( I will apply your suggested performance optimizations to the next patch.
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Assignee: Jochen Wiedmann
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Commented: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Robert Eric Reeves (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12523718 ]
Robert Eric Reeves commented on JAXME-94:
-----------------------------------------
The following is an email from Jose that describes his approach to his patch:
I sent you an attachment with the patch that solves the problem of the huge size in the classes that parse sequences (in addition, it solves an existing problem with multiplicities other than 0..required for minOccurs and 1..unbounded for maxOccurs, some unit test created)
As I told you yesterday, the class to parse a sequence with 250 optional elements was around 290.000 lines long before applying the optimization, and now the class to parse the same sequence is around 4.000 lines.
I have elaborated a bit more the concept of a state machine that was underlying in the existing code to validate the structure of elements in the sequence, using regular expressions.
My first approach was moving all the logic that decides if the received element is valid in the current state to a table of transitions. It worked but I still didn't like the solution, because, while the code that takes the decision was reduced dramatically, the code to initialize the table with all the states and transitions became huge, and the class was still 40.000 lines long (better than 290.000 but still not good). Besides, the table of transitions was very memory consumpting.
Then I remembered that a a deterministic finite state machine can always be represented by a regular expression and then all the table stuff could be substituted by generating an adequate RE. For example:
<xsd:sequence>
<xsd:element name="nolessthan3" minOccurs="3" maxOccurs= "unbounded" type="xsd:string"/>
<xsd:element name="nomorethan3" minOccurs="0" maxOccurs= "3" type="xsd:string"/> <xsd:element name="between3and5" minOccurs="3" maxOccurs= "5" type="xsd:string"/> </xsd:sequence>
If we assign symbols to the elements ("nolessthan3" --> 0, "nomorethan3" --> 1, "between3and5" --> 2), then a XML that follows this XSD could be validated using the regular expression 0{3,}1{0,3}2{3,5}
This works smoothly and all the tests that comes with JaxMe keep passing. I have also tried the modification with the XSDs and XMLs of our project without apparent problems. I could even have reduced more the code, but there is a lot of information about the elements that is available only at "generating classes time" but it is not at "parsing time" and changing it would be very difficult only to save a few lines.
The patch is intended to be applied over a fresh 0.5.2 or 0.6-SNAPSHOT(trunk) version (without my previous patch).
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Updated: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Jose Luis Huertas (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jose Luis Huertas updated JAXME-94:
-----------------------------------
Attachment: veryLongSequences-refactor-patch-0.5.2.diff
Patch that solves this issue refactoring the code of the startElement() method in several private methods (one for each state).
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Assignee: Jochen Wiedmann
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff, veryLongSequences-refactor-patch-0.5.2.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Updated: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Robert Eric Reeves (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Eric Reeves updated JAXME-94:
------------------------------------
Attachment: veryLongSequences-patch-0.5.2.diff
Patch for 0.5.2
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Assigned: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Jochen Wiedmann (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jochen Wiedmann reassigned JAXME-94:
------------------------------------
Assignee: Jochen Wiedmann
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Assignee: Jochen Wiedmann
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Updated: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Jose Luis Huertas (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jose Luis Huertas updated JAXME-94:
-----------------------------------
Attachment: Jaxme94-SolutionB-0.6-SNAPSHOT.diff
Jaxme94-SolutionB-0.5.3-SNAPSHOT.diff
Jaxme94-SolutionB-0.5.2.diff
Finally I haven't been able to find a replacement for the method Matcher.hitEnd() that worked in JDK 1.4 and that was the key stone of the solution so I have had to change my strategy again :(
Basically I maintain a couple of arrays with the elements associated to each state, their minOccurs, maxOccurs and current occurs during the parsing process. The length of the generated class is similar to my previous solution (the one that used regular expressions), aprox. 4.000 lines for a sequence of 250 elements against aprox. 290.000 lines that generates the current code.
In addition, it also solves the problem with multiplicities other than 0..required for minOccurs and 1..unbounded for maxOccurs.
I attach three patches, one suitable to apply to 0.5.2 version (Jaxme94-SolutionB-0.5.2.diff), another for the current 0.5 branch (Jaxme\Jaxme94-SolutionB-0.5.3-SNAPSHOT.diff) and another for the current head (Jaxme\Jaxme94-SolutionB-0.6-SNAPSHOT.diff), so it is up to you to decide the more appropiate one.
BTW, I have tried to make the arrays static for performance, as Jochen pointed out in his previous comments, but I haven't been able since sometimes the classes are generated as inner ones inside an outer element, and it seems that inner classes cannot have static members :(
Please have a look at it and if you have any problem or comments I will be happy to make the changes needed.
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Assignee: Jochen Wiedmann
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: Jaxme94-SolutionB-0.5.2.diff, Jaxme94-SolutionB-0.5.3-SNAPSHOT.diff, Jaxme94-SolutionB-0.6-SNAPSHOT.diff, test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff, veryLongSequences-refactor-patch-0.5.2.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Updated: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Robert Eric Reeves (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Eric Reeves updated JAXME-94:
------------------------------------
Attachment: test.xsd
Test XSD to generate very large startElement method
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org
[jira] Updated: (JAXME-94) startElement method larger than 64KB
when using a large amount of complexType's
Posted by "Robert Eric Reeves (JIRA)" <ja...@ws.apache.org>.
[ https://issues.apache.org/jira/browse/JAXME-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Eric Reeves updated JAXME-94:
------------------------------------
Attachment: veryLongSequences-patch-0.6-SNAPSHOT.diff
Patch for 0.6
> startElement method larger than 64KB when using a large amount of complexType's
> -------------------------------------------------------------------------------
>
> Key: JAXME-94
> URL: https://issues.apache.org/jira/browse/JAXME-94
> Project: JaxMe
> Issue Type: Improvement
> Components: JaxMe Core
> Affects Versions: 0.5, 0.6
> Environment: All
> Reporter: Robert Eric Reeves
> Priority: Critical
> Fix For: 0.5, 0.6
>
> Attachments: test.xsd, veryLongSequences-patch-0.5.2.diff, veryLongSequences-patch-0.6-SNAPSHOT.diff
>
>
> Using an XSD that has a complexType with ~50 child complexTypes in a sequence, the generate Java code has a method startElement that is too large for javac.
> Jose Luis Huertas Fernández has created a patch to fix this error. I will be attaching it to this issue. There are two patches. One for 5.2 and the other for 6.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: jaxme-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: jaxme-dev-help@ws.apache.org