You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Aaron Digulla <di...@hepe.com> on 2008/08/06 09:48:11 UTC

Changes necessary in StAX to get it on par with DecentXML

Hello,

That didn't work well. Okay, since you don't believe me, here is an  
(incomplete) list of changes I would need in StAX to be able to use it  
for my work instead of having to write my own XML parser. Note: If you  
feel like asking "who would ever need that?" while reading that list,  
then my answer is that I didn't want this mail to become 100KB long,  
so I won't tell you all the reasonings which finally distilled into  
the list below. I'm working with XML for 10 years, now, I have used  
almost every XML tool out there and none of them could cover all the  
ground I need to plow.

- Remove final from class XmlReader
- The interface Attribute needs a new method which returns the raw  
value read by the parser (i.e. without &quot; expanded)
- The interface Attribute needs a new method to specify the quote  
which was read/should be used during writing
- The reader API must allow a way to return entity references  
unexpanded plus a way to expand the references during parsing and later
- The StartElement must be extended to allow to specify the order in  
which attributes and name space declarations where in the input
- A way to find out if the namespace was given by prefixing the  
element name or by using an xmlns attribute.
- There must be a special text element which can be used everywhere  
where Attribute is used.
- The writer is probably powerful enough; I'd just use writeRaw() in a  
lot of places (which kind of defeats the purpose but it would work).

Since the client API should ignore anything it doesn't understand, it  
might be possible to implement this without breaking client code but  
I'm only 80% sure of that at this time.

That would get me about 30% to where I am now with DecentXML, because  
DecentXML doesn't only read XML, it also creates a DOM-like structure  
for me, which I can navigate and modify. As far as I can see, stax  
doesn't have any support for collecting the data in a smart data  
structure.

These are the reasons why I don't think that StAX would be the right  
tool for my task, why I think that no amount of work spent on it would  
ever make it the right tool. Even if I spent the time, the result  
would be as non-standard as DecentXML and DecentXML already does what  
I need after three days of work.

I have poured the wisdom of 10 years working with various XML parsers  
and tool and that is why I'm so confident that no other XML tool can  
do what it can.

I'm not 100% sure whether StAX could be modified to allow to  
modify/filter POM files without too much disruption, because POM uses  
only a small subset of XML. But then, Maven POMs are only 1% of my  
daily work and I need a tool that can solve the other 99% as well.

A lot of people find that XML can do whatever they need but when I  
asked them, after a bit of poking, the real answer is that they  
actually needed something else but gave up on it, because they  
eventually realized that no tool on the market gave them the necessary  
flexibility to do it. So again, the available tools have limited what  
they can do instead of the other way around. That is the issue  
DecentXML tries to solve.

Regards,

-- 
Aaron "Optimizer" Digulla a.k.a. Philmann Dark
"It's not the universe that's limited, it's our imagination.
Follow me and I'll show you something beyond the limits."
http://www.pdark.de/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: Changes necessary in StAX to get it on par with DecentXML

Posted by Stephen Connolly <st...@gmail.com>.
For round-triping, I am 99.99% certain that no other changes are required to
StAX (providing you are fine with the Event half of the API)

For generating XML to match exacting formatting, a special mode whereby
ignorableSpace events can be raised and received in between Attribute
events.

There could be a need some other event required to signal the end of the
start tag.

-Stephen

On Sat, Aug 9, 2008 at 12:58 AM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:

> OK... one change needed in the StAX API is to fix
> XMLOutputFactory.newInstance(String, ClassLoader) to return XMLOutputFactory
> and not XMLInputFactory.
>
> ;-)
>
>
> On Fri, Aug 8, 2008 at 7:55 PM, Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
>> On Fri, Aug 8, 2008 at 2:15 PM, Aaron Digulla <di...@hepe.com> wrote:
>>
>>> Quoting Jochen Wiedmann <jo...@gmail.com>:
>>>
>>>  That didn't work well. Okay, since you don't believe me, here is an
>>>>> (incomplete) list of changes I would need in StAX to be able to use it
>>>>> for
>>>>> my work instead of having to write my own XML parser.
>>>>>
>>>>
>>>> I think here is a misunderstandment. The question was not (at least
>>>> not IMO) whether you could rewrite DecentXML to use StAX, but whether
>>>> you could give DecentXML a StAX API. That way Maven components could
>>>> basically trust in a standard API, except for the cases where the
>>>> differences matter.
>>>>
>>>
>>> Internally, DecentXML uses a StAX-like API (the DOM parser part uses a
>>> tokenizer to break the input into pieces). The problem is that I can't base
>>> DecentXML on StAX because StAX throws information away that I need and I
>>> can't offer a StAX-compatible API because StAX isn't meant to keep the
>>> information I need to preserve.
>>>
>>> I could create a filter class which strips the information that DecentXML
>>> provides down to a StAX API but that would still be useless:
>>>
>>> 1. Case: You read POM files to create an object model. Here, you would
>>> gain nothing except you'd have to look for new bugs.
>>>
>>> 2. Case: You read POM files to filter them. Here the StAX API is useless
>>> because it throws too much information away, so this code would have to be
>>> written from scratch using an incompatible API anyway, no matter how you
>>> solve it.
>>>
>>> That said, I've written a Maven search'r'replace tool using DecentXML.
>>> It's standalone (no dependencies besides DecentXML and Java's rt.jar), it
>>> can search pom.xml files with certain elements/texts and on these files, it
>>> can print certain parts (search), check that certain parts have certain
>>> values (for example, that all parent elements have the right version in
>>> them) and it can replace existing values with new ones.
>>>
>>> The source is roughly 400 lines:
>>> http://code.google.com/p/decentxml/source/browse/trunk/src/test/java/de/pdark/decentxml/MavenSNR.java?r=34
>>>
>>> What I would like to see is that Maven keeps the StAX API internally to
>>> build the POM and to do its job. Only when a plugin needs to *manipulate*
>>> XML that a *user* has written *by hand*, it should use DecentXML. Currently,
>>> I know only of these plugins which fall into that category:
>>>
>>> - archetype
>>> - war
>>> - version
>>>
>>> All other plugins wouldn't gain enough from a transition to make the
>>> effort worthwhile.
>>>
>>
>> I actually think that this can be done... but only using the
>> EventReader/EventWriter model.
>>
>> All the XMLEvents returned by the XMLEventReader are just interfaces.
>>
>> You store in a concrete class for each type of event the information you
>> need.  In fact I'm writing an implementation just now.
>>
>> The XMLEventWriter then takes the events back... if they are the events
>> that originated from my XMLEventReader, then I just use the exact padding
>> and prefixing, etc that was read in... if they are new events that
>> originated from elsewhere, I examine how previous events were formatted and
>> try to replicate that.
>>
>> I agree that the cursor interface of StAX will not do what we need... but
>> the Event interface can, as I suspected... but needed to confirm, do exactly
>> what we need... and I am in the process of writing an implementation.
>>
>> The only issue that I could see blocking me is how the StAX specification
>> defines the handling of CR, LF and CR/LF pairs. I suspect that I can stash
>> this information when reading provided the XMLEventReader is constructed
>> from the _correct_ stream.
>>
>> -Stephen.
>>
>>
>>
>>>
>>>
>>> Regards,
>>>
>>> --
>>> Aaron "Optimizer" Digulla a.k.a. Philmann Dark
>>> "It's not the universe that's limited, it's our imagination.
>>> Follow me and I'll show you something beyond the limits."
>>> http://www.pdark.de/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>
>>>
>>
>

Re: Changes necessary in StAX to get it on par with DecentXML

Posted by Stephen Connolly <st...@gmail.com>.
OK... one change needed in the StAX API is to fix
XMLOutputFactory.newInstance(String, ClassLoader) to return XMLOutputFactory
and not XMLInputFactory.

;-)

On Fri, Aug 8, 2008 at 7:55 PM, Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:

> On Fri, Aug 8, 2008 at 2:15 PM, Aaron Digulla <di...@hepe.com> wrote:
>
>> Quoting Jochen Wiedmann <jo...@gmail.com>:
>>
>>  That didn't work well. Okay, since you don't believe me, here is an
>>>> (incomplete) list of changes I would need in StAX to be able to use it
>>>> for
>>>> my work instead of having to write my own XML parser.
>>>>
>>>
>>> I think here is a misunderstandment. The question was not (at least
>>> not IMO) whether you could rewrite DecentXML to use StAX, but whether
>>> you could give DecentXML a StAX API. That way Maven components could
>>> basically trust in a standard API, except for the cases where the
>>> differences matter.
>>>
>>
>> Internally, DecentXML uses a StAX-like API (the DOM parser part uses a
>> tokenizer to break the input into pieces). The problem is that I can't base
>> DecentXML on StAX because StAX throws information away that I need and I
>> can't offer a StAX-compatible API because StAX isn't meant to keep the
>> information I need to preserve.
>>
>> I could create a filter class which strips the information that DecentXML
>> provides down to a StAX API but that would still be useless:
>>
>> 1. Case: You read POM files to create an object model. Here, you would
>> gain nothing except you'd have to look for new bugs.
>>
>> 2. Case: You read POM files to filter them. Here the StAX API is useless
>> because it throws too much information away, so this code would have to be
>> written from scratch using an incompatible API anyway, no matter how you
>> solve it.
>>
>> That said, I've written a Maven search'r'replace tool using DecentXML.
>> It's standalone (no dependencies besides DecentXML and Java's rt.jar), it
>> can search pom.xml files with certain elements/texts and on these files, it
>> can print certain parts (search), check that certain parts have certain
>> values (for example, that all parent elements have the right version in
>> them) and it can replace existing values with new ones.
>>
>> The source is roughly 400 lines:
>> http://code.google.com/p/decentxml/source/browse/trunk/src/test/java/de/pdark/decentxml/MavenSNR.java?r=34
>>
>> What I would like to see is that Maven keeps the StAX API internally to
>> build the POM and to do its job. Only when a plugin needs to *manipulate*
>> XML that a *user* has written *by hand*, it should use DecentXML. Currently,
>> I know only of these plugins which fall into that category:
>>
>> - archetype
>> - war
>> - version
>>
>> All other plugins wouldn't gain enough from a transition to make the
>> effort worthwhile.
>>
>
> I actually think that this can be done... but only using the
> EventReader/EventWriter model.
>
> All the XMLEvents returned by the XMLEventReader are just interfaces.
>
> You store in a concrete class for each type of event the information you
> need.  In fact I'm writing an implementation just now.
>
> The XMLEventWriter then takes the events back... if they are the events
> that originated from my XMLEventReader, then I just use the exact padding
> and prefixing, etc that was read in... if they are new events that
> originated from elsewhere, I examine how previous events were formatted and
> try to replicate that.
>
> I agree that the cursor interface of StAX will not do what we need... but
> the Event interface can, as I suspected... but needed to confirm, do exactly
> what we need... and I am in the process of writing an implementation.
>
> The only issue that I could see blocking me is how the StAX specification
> defines the handling of CR, LF and CR/LF pairs. I suspect that I can stash
> this information when reading provided the XMLEventReader is constructed
> from the _correct_ stream.
>
> -Stephen.
>
>
>
>>
>>
>> Regards,
>>
>> --
>> Aaron "Optimizer" Digulla a.k.a. Philmann Dark
>> "It's not the universe that's limited, it's our imagination.
>> Follow me and I'll show you something beyond the limits."
>> http://www.pdark.de/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>

Re: Changes necessary in StAX to get it on par with DecentXML

Posted by Stephen Connolly <st...@gmail.com>.
On Fri, Aug 8, 2008 at 2:15 PM, Aaron Digulla <di...@hepe.com> wrote:

> Quoting Jochen Wiedmann <jo...@gmail.com>:
>
>  That didn't work well. Okay, since you don't believe me, here is an
>>> (incomplete) list of changes I would need in StAX to be able to use it
>>> for
>>> my work instead of having to write my own XML parser.
>>>
>>
>> I think here is a misunderstandment. The question was not (at least
>> not IMO) whether you could rewrite DecentXML to use StAX, but whether
>> you could give DecentXML a StAX API. That way Maven components could
>> basically trust in a standard API, except for the cases where the
>> differences matter.
>>
>
> Internally, DecentXML uses a StAX-like API (the DOM parser part uses a
> tokenizer to break the input into pieces). The problem is that I can't base
> DecentXML on StAX because StAX throws information away that I need and I
> can't offer a StAX-compatible API because StAX isn't meant to keep the
> information I need to preserve.
>
> I could create a filter class which strips the information that DecentXML
> provides down to a StAX API but that would still be useless:
>
> 1. Case: You read POM files to create an object model. Here, you would gain
> nothing except you'd have to look for new bugs.
>
> 2. Case: You read POM files to filter them. Here the StAX API is useless
> because it throws too much information away, so this code would have to be
> written from scratch using an incompatible API anyway, no matter how you
> solve it.
>
> That said, I've written a Maven search'r'replace tool using DecentXML. It's
> standalone (no dependencies besides DecentXML and Java's rt.jar), it can
> search pom.xml files with certain elements/texts and on these files, it can
> print certain parts (search), check that certain parts have certain values
> (for example, that all parent elements have the right version in them) and
> it can replace existing values with new ones.
>
> The source is roughly 400 lines:
> http://code.google.com/p/decentxml/source/browse/trunk/src/test/java/de/pdark/decentxml/MavenSNR.java?r=34
>
> What I would like to see is that Maven keeps the StAX API internally to
> build the POM and to do its job. Only when a plugin needs to *manipulate*
> XML that a *user* has written *by hand*, it should use DecentXML. Currently,
> I know only of these plugins which fall into that category:
>
> - archetype
> - war
> - version
>
> All other plugins wouldn't gain enough from a transition to make the effort
> worthwhile.
>

I actually think that this can be done... but only using the
EventReader/EventWriter model.

All the XMLEvents returned by the XMLEventReader are just interfaces.

You store in a concrete class for each type of event the information you
need.  In fact I'm writing an implementation just now.

The XMLEventWriter then takes the events back... if they are the events that
originated from my XMLEventReader, then I just use the exact padding and
prefixing, etc that was read in... if they are new events that originated
from elsewhere, I examine how previous events were formatted and try to
replicate that.

I agree that the cursor interface of StAX will not do what we need... but
the Event interface can, as I suspected... but needed to confirm, do exactly
what we need... and I am in the process of writing an implementation.

The only issue that I could see blocking me is how the StAX specification
defines the handling of CR, LF and CR/LF pairs. I suspect that I can stash
this information when reading provided the XMLEventReader is constructed
from the _correct_ stream.

-Stephen.



>
>
> Regards,
>
> --
> Aaron "Optimizer" Digulla a.k.a. Philmann Dark
> "It's not the universe that's limited, it's our imagination.
> Follow me and I'll show you something beyond the limits."
> http://www.pdark.de/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>

Re: Changes necessary in StAX to get it on par with DecentXML

Posted by Aaron Digulla <di...@hepe.com>.
Quoting Jochen Wiedmann <jo...@gmail.com>:

>> That didn't work well. Okay, since you don't believe me, here is an
>> (incomplete) list of changes I would need in StAX to be able to use it for
>> my work instead of having to write my own XML parser.
>
> I think here is a misunderstandment. The question was not (at least
> not IMO) whether you could rewrite DecentXML to use StAX, but whether
> you could give DecentXML a StAX API. That way Maven components could
> basically trust in a standard API, except for the cases where the
> differences matter.

Internally, DecentXML uses a StAX-like API (the DOM parser part uses a  
tokenizer to break the input into pieces). The problem is that I can't  
base DecentXML on StAX because StAX throws information away that I  
need and I can't offer a StAX-compatible API because StAX isn't meant  
to keep the information I need to preserve.

I could create a filter class which strips the information that  
DecentXML provides down to a StAX API but that would still be useless:

1. Case: You read POM files to create an object model. Here, you would  
gain nothing except you'd have to look for new bugs.

2. Case: You read POM files to filter them. Here the StAX API is  
useless because it throws too much information away, so this code  
would have to be written from scratch using an incompatible API  
anyway, no matter how you solve it.

That said, I've written a Maven search'r'replace tool using DecentXML.  
It's standalone (no dependencies besides DecentXML and Java's rt.jar),  
it can search pom.xml files with certain elements/texts and on these  
files, it can print certain parts (search), check that certain parts  
have certain values (for example, that all parent elements have the  
right version in them) and it can replace existing values with new ones.

The source is roughly 400 lines:  
http://code.google.com/p/decentxml/source/browse/trunk/src/test/java/de/pdark/decentxml/MavenSNR.java?r=34

What I would like to see is that Maven keeps the StAX API internally  
to build the POM and to do its job. Only when a plugin needs to  
*manipulate* XML that a *user* has written *by hand*, it should use  
DecentXML. Currently, I know only of these plugins which fall into  
that category:

- archetype
- war
- version

All other plugins wouldn't gain enough from a transition to make the  
effort worthwhile.

Regards,

-- 
Aaron "Optimizer" Digulla a.k.a. Philmann Dark
"It's not the universe that's limited, it's our imagination.
Follow me and I'll show you something beyond the limits."
http://www.pdark.de/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: Changes necessary in StAX to get it on par with DecentXML

Posted by Jochen Wiedmann <jo...@gmail.com>.
Hi, Aaron,

On Wed, Aug 6, 2008 at 9:48 AM, Aaron Digulla <di...@hepe.com> wrote:

> That didn't work well. Okay, since you don't believe me, here is an
> (incomplete) list of changes I would need in StAX to be able to use it for
> my work instead of having to write my own XML parser.

I think here is a misunderstandment. The question was not (at least
not IMO) whether you could rewrite DecentXML to use StAX, but whether
you could give DecentXML a StAX API. That way Maven components could
basically trust in a standard API, except for the cases where the
differences matter.

Jochen


-- 
Look, that's why there's rules, understand? So that you think before
you break 'em.

 -- (Terry Pratchett, Thief of Time)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org