You are viewing a plain text version of this content. The canonical link for it is here.
Posted to j-users@xerces.apache.org by Massimo Valla <ma...@gmail.com> on 2006/03/01 21:21:01 UTC
still unsolved: parsing multiple XML from socket
Hi All,
I wish to thank all the people for the replies and suggestions about reading
multiple XML from a socket (see thread:
http://mail-archives.apache.org/mod_mbox/xerces-j-users/200602.mbox/%3cd75357300602041854y68dee379yc2e8f467bd0dcec1@mail.gmail.com%3e
)
Unfortunately no suggestion was useful for our case in which:
1- Server and protocol ***cannot be changed***. Any solution that affects
the protocol (markers, parent root tag, etc.) is *not applicable* to our
case.
2- XML messages shall be considered finished when the root tag is closed;
solutions that wait for the next <?xml to consider the XML message finished
are not applicable beacuse the server would send XML messages with some time
delay between them
The funny thing is that the solution would be exactly the same as in FAQ-11
in Xerces1 (http://xerces.apache.org/xerces-j/faq-write.html#faq-11).
The fact that a solution to the problem was given as a FAQ means that many
people were facing the same problem we have.
FAQ-11 was suggesting to solve in 3 steps:
STEP-1. Avoid the buffering of the parser by subclassing
org.apache.xerces.readers.DefaultReaderFactory
STEP-2. Terminate the SAX parse when the root tag is closed by throwing a
SAXException
STEP-3. Preventing the parser from closing the socket -> subclass
BufferedReader to provide an empty close method
*FAQ-11 was giving exactly the same solution we are looking for.*
*But unfortunately STEP-1 is not applicable to Xerces2, as the internal
class org.apache.xerces.readers.DefaultReaderFactory has been moved.*
Is there a way to avoid the buffering of the parser in Xerces2 ??
Many thanks for your help,
Massimo
Re: still unsolved: parsing multiple XML from socket
Posted by Massimo Valla <ma...@gmail.com>.
Hi Pete & Alek,
thanks for the discussion and suggestions.
I think I have enough info to try to implement a custom reader that:
1. overrides the close() to avoid the SAX parser close the socket on
exception (I still cannot understand why a parser that has received a socket
as parapmeter, has the right to close it event if he did not open it... it
is really an awful behavior to my understanding)
2. adds a doClose() { super.close(); } to acctually allow the client
to close the socket
3. overrides the read(char[] ...) to return data available on the
socket until the next '>' (will try the suggested optimization in a second
phase)
I will post my code on this forum to get feedback. Possibly it could become
a new FAQ for Xerces2 (similar to FAQ-11 for Xerces1).
For the moment, thanks for your help! Any other suggestion for the reader
implementation is more than welcome.
Bye,
Massimo
On 3/6/06, Peter Hendry <pe...@capeclear.com> wrote:
>
>
> The issue is stopping Xerces from reading ahead in the buffer. If SAX is
> used then the endElement with depth == 0 won't help if Xerces has already
> pulled in bytes of the next XML document in the stream. That was the
> original problem. So using SAX alone is not the solution.
>
> Pete
>
> --------------------------------------------------------------------- To
> unsubscribe, e-mail: j-users-unsubscribe@xerces.apache.org For additional
> commands, e-mail: j-users-help@xerces.apache.org
>
Re: still unsolved: parsing multiple XML from socket
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Peter Hendry wrote:
>
>
> Aleksander Slominski wrote:
>> Peter Hendry wrote:
>>
>>> Aleksander Slominski wrote:
>>>
>>>> Peter Hendry wrote:
>>>>
>>>>
>>>>> More efficient still would be to keep track of the last '<' and
>>>>> whether it had a '/' after it - then allow returning past '>' if the
>>>>> last '<' didn't have a '/' (confused? :-) ).
>>>>>
>>>>>
>>>> not sure if it is going to work with CDATA section (they can contain
>>>> "unbalanced" XML)
>>>>
>>>>
>>> Why not? It is just an optimization. You can't have CDATA in an end
>>> tag so you should not go past the '>' of the end tag whether there is
>>> CDATA or not.
>>>
>> CDATA section can have unbalanced XML so beynd tracking of < and /> you
>> need to track CDATA sections as well AFACS.
>>
>>
>
> This is not true. From previous mails I would assume that a SAX parser
> is being used.
where did this assumption come from (i must have missed it in the long
thread :) ) - if you look on the top post it talks about tracking '<'
and '</' ... there is no need to track them if you use XML parser (such
as SAX API) then you just need to track indentationLevel (startElement:
++indentationLevel; endElement: --indentationLevel;
if(indentationLevel==0) { parser is on the last end element };)
> In that you will received startElement and endElement events. It is
> those that are being matched up. What is in the CDATA doesn't matter.
> Until you get the endElement that matches the first startElement you
> will continue to honor read() requests but always return up to the
> next '>' (optionally optimizing with '</' checking as well). If a
> CDATA contains a '</' or any unmatched '<' or '>' it doesn't matter as
> you will read past them on the next read. Within a real end tag
> (outside cdata) you cannot get any other '>' or '<' so there is no
> problem.
>
> The optimisation would also have to account for empty tags '<x/>'.
>
> An example,
>
> <root><x><![[CDATA[<x<t/>///>>>/></x>]]></x></root>
>
> Without optimization, the reads would return the following
>
> <root>
> <x>
> <![[CDATA[<x<t/>
> ///>
> >
> >
> />
> </x>
> ]]>
> </x>
> </root>
>
> at which point the endElement even would return the depth back to 0
> and so it is known it is the end of the document.
>
> With some optimization - return if '>' and '/' has been seen since
> last '<'
>
> <root><x><![[CDATA[<x<t/>
> ///>
> >>/>
> </x>
> ]]></x>
> </root>
>
> and again at this point endElement is called and returns the depth to
> 0 so the end of the document has been reached.
>
> I still don't see the need to track CDATA in this?
if you use XML parser that is true as it deals with actual _parsing_.
tracking < and </ can be much faster though in some situations when you
pre-parse or scan input.
best,
alek
--
The best way to predict the future is to invent it - Alan Kay
---------------------------------------------------------------------
To unsubscribe, e-mail: j-users-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-users-help@xerces.apache.org
Re: still unsolved: parsing multiple XML from socket
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Peter Hendry wrote:
>
>
> Aleksander Slominski wrote:
>> Peter Hendry wrote:
>>
>>> More efficient still would be to keep track of the last '<' and
>>> whether it had a '/' after it - then allow returning past '>' if the
>>> last '<' didn't have a '/' (confused? :-) ).
>>>
>> not sure if it is going to work with CDATA section (they can contain
>> "unbalanced" XML)
>>
>
> Why not? It is just an optimization. You can't have CDATA in an end
> tag so you should not go past the '>' of the end tag whether there is
> CDATA or not.
CDATA section can have unbalanced XML so beynd tracking of < and /> you
need to track CDATA sections as well AFACS.
best,
alek
>
> Pete
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: j-users-unsubscribe@xerces.apache.org For
> additional commands, e-mail: j-users-help@xerces.apache.org
--
The best way to predict the future is to invent it - Alan Kay
---------------------------------------------------------------------
To unsubscribe, e-mail: j-users-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-users-help@xerces.apache.org
Re: still unsolved: parsing multiple XML from socket
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Peter Hendry wrote:
> To make this a bit more efficient, make read return only the chars up
> to the next '>'.
that should work. also one can still buffer underlying input in large
chunk and then return char-by-char (or smaller chunks as you describe)
to xml parser.
> More efficient still would be to keep track of the last '<' and
> whether it had a '/' after it - then allow returning past '>' if the
> last '<' didn't have a '/' (confused? :-) ).
not sure if it is going to work with CDATA section (they can contain
"unbalanced" XML)
>
> This working would depend on Xerces not doing read-ahead until it has
> reached the end of its previous read.
of course but from last time i looked inside X2 it should work ...
best,
alek
>
> Aleksander Slominski wrote:
>> Massimo Valla wrote:
>>
>>> Hi All,
>>> I wish to thank all the people for the replies and suggestions about
>>> reading multiple XML from a socket (see thread:
>>> http://mail-archives.apache.org/mod_mbox/xerces-j-users/200602.mbox/%3cd75357300602041854y68dee379yc2e8f467bd0dcec1@mail.gmail.com%3e)
>>>
>>> Unfortunately no suggestion was useful for our case in which:
>>>
>>> 1- Server and protocol ***cannot be changed***. Any solution that
>>> affects the protocol (markers, parent root tag, etc.) is *not
>>> applicable* to our case.
>>>
>>> 2- XML messages shall be considered finished when the root tag is
>>> closed; solutions that wait for the next <?xml to consider the XML
>>> message finished are not applicable beacuse the server would send XML
>>> messages with some time delay between them
>>>
>>> The funny thing is that the solution would be exactly the same as in
>>> FAQ-11 in Xerces1
>>> (http://xerces.apache.org/xerces-j/faq-write.html#faq-11 ).
>>> The fact that a solution to the problem was given as a FAQ means that
>>> many people were facing the same problem we have.
>>>
>>> FAQ-11 was suggesting to solve in 3 steps:
>>> STEP-1. Avoid the buffering of the parser by subclassing
>>> org.apache.xerces.readers.DefaultReaderFactory
>>> STEP-2. Terminate the SAX parse when the root tag is closed by
>>> throwing a SAXException
>>> STEP-3. Preventing the parser from closing the socket -> subclass
>>> BufferedReader to provide an empty close method
>>>
>>> *FAQ-11 was giving exactly the same solution we are looking for.*
>>> *But unfortunately STEP-1 is not applicable to Xerces2, as the
>>> internal class org.apache.xerces.readers.DefaultReaderFactory has been
>>> moved.*
>>>
>>> Is there a way to avoid the buffering of the parser in Xerces2 ??
>>>
>>>
>> hi Massimo,
>>
>> i think you still can do it without modifying X2 - juts make sure that
>> you create a reader class that will returns at most one character in
>> read(char[] ...) and its close() method does nothing - i am pretty sure
>> this will force X2 to read character by character. it is incredibly
>> inefficient - like making car that goes 60mph to go 0.6mph instead but
>> should do work (it would be interesting to know ow much slower it is to
>> use such reader in X2 ...)
>>
>> HTH,
>>
>> alek
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: j-users-unsubscribe@xerces.apache.org For
> additional commands, e-mail: j-users-help@xerces.apache.org
--
The best way to predict the future is to invent it - Alan Kay
---------------------------------------------------------------------
To unsubscribe, e-mail: j-users-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-users-help@xerces.apache.org
Re: still unsolved: parsing multiple XML from socket
Posted by Aleksander Slominski <as...@cs.indiana.edu>.
Massimo Valla wrote:
> Hi All,
> I wish to thank all the people for the replies and suggestions about
> reading multiple XML from a socket (see thread:
> http://mail-archives.apache.org/mod_mbox/xerces-j-users/200602.mbox/%3cd75357300602041854y68dee379yc2e8f467bd0dcec1@mail.gmail.com%3e)
>
> Unfortunately no suggestion was useful for our case in which:
>
> 1- Server and protocol ***cannot be changed***. Any solution that
> affects the protocol (markers, parent root tag, etc.) is *not
> applicable* to our case.
>
> 2- XML messages shall be considered finished when the root tag is
> closed; solutions that wait for the next <?xml to consider the XML
> message finished are not applicable beacuse the server would send XML
> messages with some time delay between them
>
> The funny thing is that the solution would be exactly the same as in
> FAQ-11 in Xerces1
> (http://xerces.apache.org/xerces-j/faq-write.html#faq-11 ).
> The fact that a solution to the problem was given as a FAQ means that
> many people were facing the same problem we have.
>
> FAQ-11 was suggesting to solve in 3 steps:
> STEP-1. Avoid the buffering of the parser by subclassing
> org.apache.xerces.readers.DefaultReaderFactory
> STEP-2. Terminate the SAX parse when the root tag is closed by
> throwing a SAXException
> STEP-3. Preventing the parser from closing the socket -> subclass
> BufferedReader to provide an empty close method
>
> *FAQ-11 was giving exactly the same solution we are looking for.*
> *But unfortunately STEP-1 is not applicable to Xerces2, as the
> internal class org.apache.xerces.readers.DefaultReaderFactory has been
> moved.*
>
> Is there a way to avoid the buffering of the parser in Xerces2 ??
>
hi Massimo,
i think you still can do it without modifying X2 - juts make sure that
you create a reader class that will returns at most one character in
read(char[] ...) and its close() method does nothing - i am pretty sure
this will force X2 to read character by character. it is incredibly
inefficient - like making car that goes 60mph to go 0.6mph instead but
should do work (it would be interesting to know ow much slower it is to
use such reader in X2 ...)
HTH,
alek
--
The best way to predict the future is to invent it - Alan Kay
---------------------------------------------------------------------
To unsubscribe, e-mail: j-users-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-users-help@xerces.apache.org