You are viewing a plain text version of this content. The canonical link for it is here.
Posted to j-dev@xerces.apache.org by Ed Staub <es...@mediaone.net> on 2000/07/28 17:12:40 UTC
Pull-model parser
Is there a pull-model API somewhere that would help ground the discussion?
I understand the high concept, but the design seems pretty murky to me.
If not, should I start working something up?
-Ed Staub
Re: Pull-model parser
Posted by Costin Manolache <co...@eng.sun.com>.
> Having a switch statement based on the type and performing some
> action is no different than registering a handler that receives
> callbacks at each "pull". And I think it's easier compared to
> the XMLHolder because the handling mechanism is the same whether
> you pull the events or if the parser notifies you of all of the
> events.
Both methods are ok - they allow getting the next "piece" from
the XML file and process it.
I don't agree it's simpler to write callbacks than it is to use a
switch. Also, I don't think the 2 methods are equivalent -
you can generate callbacks from the switch, but if you
do callbacks you can't provide the same functionality
as in switch ( each callback is in a different context, stack,
etc, local variables don't work).
In some cases it may be much simpler ( and maybe faster ) to
use the switch() method. I don't know - it's worth
exploring both alternatives.
Costin
Re: Pull-model parser
Posted by Andy Clark <an...@apache.org>.
Costin Manolache wrote:
> I think it would be much cleaner and simpler to not use the callbacks.
>
> Something like:
>
> void parseNext( XMLHolder h )
>
> Where XMLHolder will have a getType() = ( TAG, ATTRIBUTE, CDATA, etc).
>
> XMLHolder {
> int getType();
> XMLString getTag();
> XMLString getAttributeName();
> XMLString getAttributeValue();
> CharChunk getCdata(); ( or getChars(), getOffset(), getLength() )
> }
Having a switch statement based on the type and performing some
action is no different than registering a handler that receives
callbacks at each "pull". And I think it's easier compared to
the XMLHolder because the handling mechanism is the same whether
you pull the events or if the parser notifies you of all of the
events.
--
Andy Clark * IBM, JTC - Silicon Valley * andyc@apache.org
Re: Pull-model parser
Posted by Costin Manolache <co...@eng.sun.com>.
Hi,
This is my second attempt to reply to this thread, it seems my mail
has some problems.
> In a pull-parser, the application is in control but there *still*
> are callbacks. The reason is because you don't know what is in the
> content of the XML document.
I read your original proposal and this is the biggest problem I have.
I think it would be much cleaner and simpler to not use the callbacks.
Something like:
void parseNext( XMLHolder h )
Where XMLHolder will have a getType() = ( TAG, ATTRIBUTE, CDATA, etc).
XMLHolder {
int getType();
XMLString getTag();
XMLString getAttributeName();
XMLString getAttributeValue();
CharChunk getCdata(); ( or getChars(), getOffset(), getLength() )
}
A variation would be to return all attributes ( i.e. the first
parseNext() will
parse next element - if it's a tag it will return the tag name, after
that
will return the attributes, etc).
We can discuss granularity - either use the same granularity as in SAX,
or provide a second parameter to indicate how much it should parse.
The user can do something like:
parseNext( h )
if( h.getType() == TAG ) String t=h.getTag();
etc.
The code may be a bit more complex, but I think mixing
callbacks and pull is not the best idea.
( Another usefull method would be:
skipTo( XMLHolder ) that will discard everything until the desired
element.
)
Costin
>
>
> > getIntoNextElement(ns, elementName);
>
> What if the next thing is text? Would this just skip it? Would it
> throw an exception? I think it's still better to have callbacks.
>
> --
> Andy Clark * IBM, JTC - Silicon Valley * andyc@apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-dev-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-dev-help@xml.apache.org
RE: Pull-model parser
Posted by Ed Staub <es...@mediaone.net>.
Andy Clark wrote:
>Okay, but you could always build this on top of the SAX callbacks
>or the Xerces callbacks.
With callbacks, the stack cannot be used to maintain the current state of
(client) processing; it has to be kept in some storage which is external to
the stack.
I thought (without much foundation!!!) that the use case was one in which
the client's stack could model (perhaps loosely) the path from the document
root to the current context. If a developer looked at a call stack in a
debugger, it would reflect that path. That's what I was trying to get
across with the code fragment.
In a constrained schema without mixed content, top-down processing is an
easier paradigm for a client developer to work in than an event-based stream
model, although it certainly means that the parser's perceived performance
might suffer; now _it_ can't maintain its input state on the stack.
Has anyone found a URL with more detail on what Microsoft is thinking about
in this area? All I've found are the slides from the PDC.
-Ed Staub
-----Original Message-----
From: Andy Clark [mailto:andyc@apache.org]
Sent: Monday, July 31, 2000 7:53 PM
To: xerces-j-dev@xml.apache.org
Subject: Re: Pull-model parser
Ed Staub wrote:
> I was thinking of it from the point of view of an application
> which is processing the information set top-down, and wants to maintain
its
> context on the stack, e.g.,
>
> getCountry()
> {...
> getState();
> ...
> }
> getState()
> {...
> getCity();
> ...
> }
>
> etc.
>
> When unexpected content is encountered, I'd raise an exception.
Okay, but you could always build this on top of the SAX callbacks
or the Xerces callbacks.
Re: Pull-model parser
Posted by Andy Clark <an...@apache.org>.
Ed Staub wrote:
> I was thinking of it from the point of view of an application
> which is processing the information set top-down, and wants to maintain its
> context on the stack, e.g.,
>
> getCountry()
> {...
> getState();
> ...
> }
> getState()
> {...
> getCity();
> ...
> }
>
> etc.
>
> When unexpected content is encountered, I'd raise an exception.
Okay, but you could always build this on top of the SAX callbacks
or the Xerces callbacks.
--
Andy Clark * IBM, JTC - Silicon Valley * andyc@apache.org
RE: Pull-model parser
Posted by Ed Staub <es...@mediaone.net>.
The pull model I was hypothesizing would only be useful with certain
schemas,
and strongly suggests/mandates use of validation.
I was thinking of it from the point of view of an application
which is processing the information set top-down, and wants to maintain its
context on the stack, e.g.,
getCountry()
{...
getState();
...
}
getState()
{...
getCity();
...
}
etc.
When unexpected content is encountered, I'd raise an exception.
-ed
-----Original Message-----
From: Andy Clark [mailto:andyc@apache.org]
Sent: Monday, July 31, 2000 11:39 AM
To: xerces-j-dev@xml.apache.org
Subject: Re: Pull-model parser
Ed Staub wrote:
> I was picturing an API which would allow the caller to remain in
> control of the flow of execution. In other words, no callbacks.
> A caller which wanted to parse something like:
In a pull-parser, the application is in control but there *still*
are callbacks. The reason is because you don't know what is in the
content of the XML document.
> getIntoNextElement(ns, elementName);
What if the next thing is text? Would this just skip it? Would it
throw an exception? I think it's still better to have callbacks.
--
Andy Clark * IBM, JTC - Silicon Valley * andyc@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-dev-help@xml.apache.org
Re: Pull-model parser
Posted by Andy Clark <an...@apache.org>.
Ed Staub wrote:
> I was picturing an API which would allow the caller to remain in
> control of the flow of execution. In other words, no callbacks.
> A caller which wanted to parse something like:
In a pull-parser, the application is in control but there *still*
are callbacks. The reason is because you don't know what is in the
content of the XML document.
> getIntoNextElement(ns, elementName);
What if the next thing is text? Would this just skip it? Would it
throw an exception? I think it's still better to have callbacks.
--
Andy Clark * IBM, JTC - Silicon Valley * andyc@apache.org
RE: Pull-model parser
Posted by Ed Staub <es...@mediaone.net>.
Thanks, Andy.
Wow! I'm glad I asked... that's not what I thought was meant at all.
Here's what I _thought_ people were talking about.
I am _not_ advocating this; I'm merely trying to make sure that we're all
somewhere near the same page.
If this is close to what some pull-model advocates were envisioning,
_speak up_!
I was picturing an API which would allow the caller to remain in control of
the flow of execution. In other words, no callbacks. A caller which wanted
to parse something like:
<where:places xmlns:where="uri:erewhon">
<where:country where:id="USA">
<where:state where:id="Connecticut">
<where:city where:id="Hartford">
<where:addr where:id="123 Easy St."/>
</where:city>
</where:state>
</where:country>
<where:country where:id="USA">
<where:state where:id="Utah">
<where:city where:id="Salt Lake City">
<where:addr where:id="456 Hard St."/>
</where:city>
</where:state>
</where:country>
</where:places
might write something like this, with lots of looping, error handling and
processing of the input added (this is just thrown together to demonstrate,
it's not a proposal!).
String ns,elementName,attributeName, idCountry, idState, idCity, idAddr;
getIntoNextElement(ns, elementName); // 'where:places'; need contents of
element (hence 'Into')
while ( getIntoElement(ns, elementName) ) // 'where:country'
{
idCountry = getAttribute(ns, attribute); // returns 'USA';
puts 'uri:erewhon' in ns,
'id' in attribute
getIntoElement(ns, elementName); // 'where:state'
idState = getAttribute(ns, attribute); // 'Connecticut'
getIntoElement(ns, elementName); // 'where:city'
idCity = getAttribute(ns, attribute); // 'Hartford'
getElement(ns, elementName); // 'where:addr'
// note not using 'getIntoElement';
// allows getting attributes and text,
// but skips inner elements
idAddr = getAttribute(ns, attribute); // '123 Easy St.'
leaveElement("uri:erewhon", "city"); // scans through </where:city>
leaveElement("uri:erewhon", "state"); // scans through </where:state>
leaveElement("uri:erewhon", "country"); // scans through </where:country>
}
This kind of API is obviously only appropriate for XML with a very
constrained, well-defined schema, where the caller has a pretty good sense
of what's coming.
Regards,
-Ed Staub
-----Original Message-----
From: Andy Clark [mailto:andyc@apache.org]
Sent: Friday, July 28, 2000 1:44 PM
To: xerces-j-dev@xml.apache.org
Subject: Re: Pull-model parser
Ed Staub wrote:
> Is there a pull-model API somewhere that would help ground the
> discussion? I understand the high concept, but the design seems
> pretty murky to me.
I incorporated thoughts of pull-parsing into the design that
I posted. You can check that out to start. Look at the scanners
package for XMLDocumentScanner and XMLDTDScanner.
Typically, when I think of "pull" I use LEX as an example. But
unlike LEX which returns different constants that represent the
tokenization of the input, my API doesn't. Instead, you just
call scanXXX() and pass in a boolean that states whether it
should complete the parse or just perform the next logical
step of the parsing.
Underneath, the implementation would be a state machine that
performs the callbacks. The current parser has this already but
it might be a little hard to follow. But at least the basic
idea is there. The granularity of the state machine is up to
us but the smaller the granularity, the slower the parser goes
due to the overhead of all those extra method invocations.
The XMLDocumentHandler, XMLDTDHandler, and other handlers show
what can be one type of granularity. The document handler has
a larger granularity very similar to SAX's handler interfaces.
Whereas the handlers for DTDs are more fine-grained for acouple
of reasons: 1) DTDs are usually smaller than documents and once
we have grammar caching won't cause a performance problem, and
2) we'll be able to expose more detail about the DTD including
the breakdown of content models and the use of parameter
entities -- something very important to editor writers.
In regards to the content model breakdown, it makes it a lot
easier to separate the scanning of the DTD and the data
structure used to represent that content model. This is a
very important point because otherwise you have to tie the
data representation to the DTD scanner which really shouldn't
have any idea how the data should be represented.
--
Andy Clark * IBM, JTC - Silicon Valley * andyc@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-dev-help@xml.apache.org
Indenting
Posted by Nick Sullivan <ni...@bikestore.com>.
I am writing an application that parses both raw (tab delimited)
and XML files, puts them into the DOM, and then optionally writes
the result to System.out or an output file.
My problem is with indenting... There seems to be an issue depending on
how the DOM is generated. If I set the indenting to where it works with
the raw data, it breaks the XML, and vice versa. By "breaks" I mean
that the indenting is either non-existent or malformed.
This produces the proper indenting with the RAW data
StringWriter stringOut = new StringWriter();
OutputFormat format = new OutputFormat(doc); //Serialize DOM
format.setIndent(4);
format.setLineWidth(0);
XMLSerializer serial = new XMLSerializer (stringOut, format);
serial.asDOMSerializer(); // As a DOM Serializer
serial.serialize(doc.getDocumentElement());
System.out.println( "STRXML = " + stringOut.toString() );
==========================
This produces the proper indenting with the XML file generated DOM
StringWriter stringOut = new StringWriter();
OutputFormat format = new OutputFormat(doc, "UTF-8", true); //Serialize
DOM
format.setPreserveSpace(true);
XMLSerializer serial = new XMLSerializer (stringOut, format);
serial.asDOMSerializer(); // As a DOM Serializer
serial.serialize(doc.getDocumentElement());
System.out.println( "STRXML = " + stringOut.toString() );
But neither one works for BOTH the raw data and the XML file. I could hack
this with an "if" statement depending on how the DOM is generated...
But I was hoping there was a more elegant solution... Anyone?
I thought it might be a problem with how I was generating the DOM
from the raw data, but I took code from the DOMGenerate sample
and it does the same thing...Nevertheless, here is the code for both:
XML file:
InputSource XMLFile = new InputSource(new FileReader(XMLFile));
DOMParser parser = new DOMParser();
parser.parse(XMLData);
Raw data:
//Set up the DOM and create the root element
Document doc= new DocumentImpl();
Element root = doc.createElement("rows");
//Get the field names from the first line
String[] FieldNames = Javortex.Split(Delimiter, in.readLine());
//Walk through the rest of the file and get FieldData
while ((DataLine=in.readLine()) != null) {
Element record = doc.createElement("row");
//Create an attribute for rowNum
record.setAttribute("rowNum", String.valueOf(rowNum++));
//Split and Print the data
String[] FieldData = Javortex.Split(Delimiter, DataLine,
FieldNames.length);
for (i=0; i < FieldNames.length; i++) {
Element column = doc.createElement(FieldNames[i]);
//Only create the TextNode if its not null, so it becomes an empty
tag
if (FieldData[i].length() > 0 )
column.appendChild(doc.createTextNode(FieldData[i]));
record.appendChild(column);
}
root.appendChild(record);
}
// Wrap up
doc.appendChild(root);
BTW, I am using Xerces 1.12...
Nick Sullivan
Re: Pull-model parser
Posted by Andy Clark <an...@apache.org>.
Ed Staub wrote:
> Is there a pull-model API somewhere that would help ground the
> discussion? I understand the high concept, but the design seems
> pretty murky to me.
I incorporated thoughts of pull-parsing into the design that
I posted. You can check that out to start. Look at the scanners
package for XMLDocumentScanner and XMLDTDScanner.
Typically, when I think of "pull" I use LEX as an example. But
unlike LEX which returns different constants that represent the
tokenization of the input, my API doesn't. Instead, you just
call scanXXX() and pass in a boolean that states whether it
should complete the parse or just perform the next logical
step of the parsing.
Underneath, the implementation would be a state machine that
performs the callbacks. The current parser has this already but
it might be a little hard to follow. But at least the basic
idea is there. The granularity of the state machine is up to
us but the smaller the granularity, the slower the parser goes
due to the overhead of all those extra method invocations.
The XMLDocumentHandler, XMLDTDHandler, and other handlers show
what can be one type of granularity. The document handler has
a larger granularity very similar to SAX's handler interfaces.
Whereas the handlers for DTDs are more fine-grained for acouple
of reasons: 1) DTDs are usually smaller than documents and once
we have grammar caching won't cause a performance problem, and
2) we'll be able to expose more detail about the DTD including
the breakdown of content models and the use of parameter
entities -- something very important to editor writers.
In regards to the content model breakdown, it makes it a lot
easier to separate the scanning of the DTD and the data
structure used to represent that content model. This is a
very important point because otherwise you have to tie the
data representation to the DTD scanner which really shouldn't
have any idea how the data should be represented.
--
Andy Clark * IBM, JTC - Silicon Valley * andyc@apache.org