You are viewing a plain text version of this content. The canonical link for it is here.
Posted to xindice-dev@xml.apache.org by JC Tchitchiama <jc...@panonet.net> on 2003/11/12 12:36:56 UTC
XMLResource and OutputStream
All,
XMLResource interface is seriously lacking and output stream method. As you
know we have many tools XMLOutputter, DOMOutputter XMLSerializer helping to
store documents in an output stream, but strangely enough XMLDB interface
(XMLResource) does not accept output stream.
In an attempt to remedie this problem I would like to submit this patch for
discussion.
Couple of things first.
1) somebody has to lobby for the XMLResource interface to be extended with a
method OutputStream setContentAsOutputStream() throws XMLDBException
2) I have attached a patch file [for sake of discussion]. I know that the
implementation can be polished off with better OutputStream classes or even a
PipedInputStream/OutputStream combinaison or even some other suggestions you
may have. I have also left out the IOException handling.
3) If this patch does not PASS the discussion phase then others can use is as
an example to get round output stream issue.
Best Regards.
JC.
\\- - -//
( @ @ )
===oOOo-(_)-oOOo=================================================
jct@panonet.net
=================================================================
Re: XMLResource and OutputStream
Posted by Vadim Gritsenko <va...@reverycodes.com>.
JC Tchitchiama wrote:
>On Saturday 20 Dec 2003 6:20 pm, Vadim Gritsenko wrote:
>
>
>>JC Tchitchiama wrote:
>>
>>
.....
>>>1) somebody has to lobby for the XMLResource interface to be extended with
>>>a method OutputStream setContentAsOutputStream() throws XMLDBException
>>>
>>>2) I have attached a patch file [for sake of discussion]. I know that the
>>>implementation can be polished off with better OutputStream classes or
>>>even a PipedInputStream/OutputStream combinaison or even some other
>>>suggestions you may have. I have also left out the IOException handling.
>>>
>>>
>>>
>>Hi,
>>
>>I see couple of issues with the patch:
>>
>>1. As far as I can see it will work only for small documents, up to 8Kb.
>>Once document is large than 8Kb, buffered out stream will write to pipe
>>stream more than once and it will override value of the this.content
>>several times, loosing data.
>>
>>
>I am aware of the limitations you mentioned but I had to get the ball rolling.
>
>
Ok, fair enough. Let's roll it ;-)
>One possible solution is use PipedOutputStream smartly and read the buffer out
>progressively. The trouble there is that if a failure appends then you have a
>partial document.
>Second possible solution could to rely on the document size. If we somehow
>get the document size first before creating the BufferedOutputStream the we
>should be OK.
>
>Any other thoughts ?
>
>
Sure. Every OutputStream has a close() method, and I think we should
rely on this, as this is a standard contract in the java.io package. So,
this patch can be changed to use new BufferedOutputStream(new
ByteArrayOutputStream()), and set the resource's content when close() is
called.
Thus, you will not have issues with the buffer size / document size, and
buffer size can be reduced to 1024 or so.
>>2. Why trim is necessary, can you provide with more details - what
>>unwanted characters, what parsers?
>>
>>
>I am xerces. When using KHexEdit I could see some charaters appended to the
>document that's why I trim the string.
>
>
Can you give specific example - what is added, what is code snippet
which writes with these extra characters? I really don't like this
trim() and I think that there is a bug elsewhere.
Vadim
>>>Index: XMLResourceImpl.java
>>>===================================================================
>>>RCS file:
>>>/home/cvspublic/xml-xindice/java/src/org/apache/xindice/client/xmldb/reso
>>>urces/XMLResourceImpl.java,v retrieving revision 1.16
>>>diff -u -r1.16 XMLResourceImpl.java
>>>--- XMLResourceImpl.java 9 Aug 2003 05:01:55 -0000 1.16
>>>+++ XMLResourceImpl.java 12 Nov 2003 11:31:09 -0000
>>>@@ -78,6 +78,11 @@
>>>import javax.xml.parsers.SAXParser;
>>>import javax.xml.parsers.SAXParserFactory;
>>>import java.io.StringReader;
>>>+import java.io.OutputStream;
>>>+import java.io.IOException;
>>>+import java.io.PipedOutputStream;
>>>+import java.io.BufferedOutputStream;
>>>+
>>>
>>>/**
>>> * XMLResourceImpl provides an implementation to handle XML resources
>>>@@ -100,6 +105,14 @@
>>> protected String content = null;
>>>
>>> /**
>>>+ * buffer size generously big
>>>+ *
>>>+ */
>>>+ public static final int BUFFER_SIZE = 8*1024;
>>>+
>>>+
>>>+
>>>+ /**
>>> * This is a SAX feature that controls how namespaces are reported in
>>>SAX. * By default this feature is <em>on</em>.
>>> *
>>>@@ -313,6 +326,40 @@
>>> */
>>> public ContentHandler setContentAsSAX() throws XMLDBException {
>>> return new SetContentHandler(this);
>>>+ }
>>>+
>>>+ /**
>>>+ * setContentAsOutputStream returns a OutputStream that can be used
>>>to set the + * content of the resource.
>>>+ *
>>>+ * @return The outputStream that is used to insert data into the
>>>database. + * @exception XMLDBException
>>>+ */
>>>+ public OutputStream setContentAsOutputStream() throws XMLDBException
>>>{ +
>>>+ PipedOutputStream pipedOut = new PipedOutputStream()
>>>+ {
>>>+ public void write(byte b[], int off, int len) throws IOException
>>>+ {
>>>+ try
>>>+ {
>>>+ /** WARNING it's important to trim the string
>>>+ * because when XMLOutputter, DOMOutputter, XMLSerializer
>>>+ * is used to populate the outputstream; they append
>>>+ * charaters unwanted by most XML parsers.
>>>+ */
>>>+ content = new String(b).trim();
>>>+ bytes = null;
>>>+ } catch (Exception e) {
>>>+ //TODO log an exception here
>>>+ //throw FaultCodes.createXMLDBException(e);
>>>+ }
>>>+
>>>+ }
>>>+
>>>+ };
>>>+
>>>+ return new BufferedOutputStream(pipedOut, BUFFER_SIZE);
>>> }
>>>
>>>
Re: XMLResource and OutputStream
Posted by JC Tchitchiama <jc...@panonet.net>.
On Saturday 20 Dec 2003 6:20 pm, Vadim Gritsenko wrote:
> JC Tchitchiama wrote:
> >All,
> >
> >XMLResource interface is seriously lacking and output stream method. As
> > you know we have many tools XMLOutputter, DOMOutputter XMLSerializer
> > helping to store documents in an output stream, but strangely enough
> > XMLDB interface (XMLResource) does not accept output stream.
> >
> >In an attempt to remedie this problem I would like to submit this patch
> > for discussion.
> >
> >Couple of things first.
> >
> >1) somebody has to lobby for the XMLResource interface to be extended with
> > a method OutputStream setContentAsOutputStream() throws XMLDBException
> >
> >2) I have attached a patch file [for sake of discussion]. I know that the
> >implementation can be polished off with better OutputStream classes or
> > even a PipedInputStream/OutputStream combinaison or even some other
> > suggestions you may have. I have also left out the IOException handling.
> >
> >3) If this patch does not PASS the discussion phase then others can use is
> > as an example to get round output stream issue.
>
> Hi,
>
> I see couple of issues with the patch:
>
> 1. As far as I can see it will work only for small documents, up to 8Kb.
> Once document is large than 8Kb, buffered out stream will write to pipe
> stream more than once and it will override value of the this.content
> several times, loosing data.
I am aware of the limitations you mentioned but I had to get the ball rolling.
One possible solution is use PipedOutputStream smartly and read the buffer out
progressively. The trouble there is that if a failure appends then you have a
partial document.
Second possible solution could to rely on the document size. If we somehow
get the document size first before creating the BufferedOutputStream the we
should be OK.
Any other thoughts ?
>
> 2. Why trim is necessary, can you provide with more details - what
> unwanted characters, what parsers?
I am xerces. When using KHexEdit I could see some charaters appended to the
document that's why I trim the string.
>
> Thanks,
> Vadim
>
> >Best Regards.
> >
> >JC.
> >
> >------------------------------------------------------------------------
> >
> >Index: XMLResourceImpl.java
> >===================================================================
> >RCS file:
> > /home/cvspublic/xml-xindice/java/src/org/apache/xindice/client/xmldb/reso
> >urces/XMLResourceImpl.java,v retrieving revision 1.16
> >diff -u -r1.16 XMLResourceImpl.java
> >--- XMLResourceImpl.java 9 Aug 2003 05:01:55 -0000 1.16
> >+++ XMLResourceImpl.java 12 Nov 2003 11:31:09 -0000
> >@@ -78,6 +78,11 @@
> > import javax.xml.parsers.SAXParser;
> > import javax.xml.parsers.SAXParserFactory;
> > import java.io.StringReader;
> >+import java.io.OutputStream;
> >+import java.io.IOException;
> >+import java.io.PipedOutputStream;
> >+import java.io.BufferedOutputStream;
> >+
> >
> > /**
> > * XMLResourceImpl provides an implementation to handle XML resources
> >@@ -100,6 +105,14 @@
> > protected String content = null;
> >
> > /**
> >+ * buffer size generously big
> >+ *
> >+ */
> >+ public static final int BUFFER_SIZE = 8*1024;
> >+
> >+
> >+
> >+ /**
> > * This is a SAX feature that controls how namespaces are reported in
> > SAX. * By default this feature is <em>on</em>.
> > *
> >@@ -313,6 +326,40 @@
> > */
> > public ContentHandler setContentAsSAX() throws XMLDBException {
> > return new SetContentHandler(this);
> >+ }
> >+
> >+ /**
> >+ * setContentAsOutputStream returns a OutputStream that can be used
> > to set the + * content of the resource.
> >+ *
> >+ * @return The outputStream that is used to insert data into the
> > database. + * @exception XMLDBException
> >+ */
> >+ public OutputStream setContentAsOutputStream() throws XMLDBException
> > { +
> >+ PipedOutputStream pipedOut = new PipedOutputStream()
> >+ {
> >+ public void write(byte b[], int off, int len) throws IOException
> >+ {
> >+ try
> >+ {
> >+ /** WARNING it's important to trim the string
> >+ * because when XMLOutputter, DOMOutputter, XMLSerializer
> >+ * is used to populate the outputstream; they append
> >+ * charaters unwanted by most XML parsers.
> >+ */
> >+ content = new String(b).trim();
> >+ bytes = null;
> >+ } catch (Exception e) {
> >+ //TODO log an exception here
> >+ //throw FaultCodes.createXMLDBException(e);
> >+ }
> >+
> >+ }
> >+
> >+ };
> >+
> >+ return new BufferedOutputStream(pipedOut, BUFFER_SIZE);
> > }
> >
> > /**
--
Best Regards.
JC.
\\- - -//
( @ @ )
===oOOo-(_)-oOOo=================================================
jct@panonet.net
=================================================================
Re: XMLResource and OutputStream
Posted by Vadim Gritsenko <va...@verizon.net>.
JC Tchitchiama wrote:
>All,
>
>XMLResource interface is seriously lacking and output stream method. As you
>know we have many tools XMLOutputter, DOMOutputter XMLSerializer helping to
>store documents in an output stream, but strangely enough XMLDB interface
>(XMLResource) does not accept output stream.
>
>In an attempt to remedie this problem I would like to submit this patch for
>discussion.
>
>Couple of things first.
>
>1) somebody has to lobby for the XMLResource interface to be extended with a
>method OutputStream setContentAsOutputStream() throws XMLDBException
>
>2) I have attached a patch file [for sake of discussion]. I know that the
>implementation can be polished off with better OutputStream classes or even a
>PipedInputStream/OutputStream combinaison or even some other suggestions you
>may have. I have also left out the IOException handling.
>
>3) If this patch does not PASS the discussion phase then others can use is as
>an example to get round output stream issue.
>
>
Hi,
I see couple of issues with the patch:
1. As far as I can see it will work only for small documents, up to 8Kb.
Once document is large than 8Kb, buffered out stream will write to pipe
stream more than once and it will override value of the this.content
several times, loosing data.
2. Why trim is necessary, can you provide with more details - what
unwanted characters, what parsers?
Thanks,
Vadim
>Best Regards.
>
>JC.
> \\- - -//
> ( @ @ )
>===oOOo-(_)-oOOo=================================================
> jct@panonet.net
>=================================================================
>
>
>------------------------------------------------------------------------
>
>Index: XMLResourceImpl.java
>===================================================================
>RCS file: /home/cvspublic/xml-xindice/java/src/org/apache/xindice/client/xmldb/resources/XMLResourceImpl.java,v
>retrieving revision 1.16
>diff -u -r1.16 XMLResourceImpl.java
>--- XMLResourceImpl.java 9 Aug 2003 05:01:55 -0000 1.16
>+++ XMLResourceImpl.java 12 Nov 2003 11:31:09 -0000
>@@ -78,6 +78,11 @@
> import javax.xml.parsers.SAXParser;
> import javax.xml.parsers.SAXParserFactory;
> import java.io.StringReader;
>+import java.io.OutputStream;
>+import java.io.IOException;
>+import java.io.PipedOutputStream;
>+import java.io.BufferedOutputStream;
>+
>
> /**
> * XMLResourceImpl provides an implementation to handle XML resources
>@@ -100,6 +105,14 @@
> protected String content = null;
>
> /**
>+ * buffer size generously big
>+ *
>+ */
>+ public static final int BUFFER_SIZE = 8*1024;
>+
>+
>+
>+ /**
> * This is a SAX feature that controls how namespaces are reported in SAX.
> * By default this feature is <em>on</em>.
> *
>@@ -313,6 +326,40 @@
> */
> public ContentHandler setContentAsSAX() throws XMLDBException {
> return new SetContentHandler(this);
>+ }
>+
>+ /**
>+ * setContentAsOutputStream returns a OutputStream that can be used to set the
>+ * content of the resource.
>+ *
>+ * @return The outputStream that is used to insert data into the database.
>+ * @exception XMLDBException
>+ */
>+ public OutputStream setContentAsOutputStream() throws XMLDBException {
>+
>+ PipedOutputStream pipedOut = new PipedOutputStream()
>+ {
>+ public void write(byte b[], int off, int len) throws IOException
>+ {
>+ try
>+ {
>+ /** WARNING it's important to trim the string
>+ * because when XMLOutputter, DOMOutputter, XMLSerializer
>+ * is used to populate the outputstream; they append
>+ * charaters unwanted by most XML parsers.
>+ */
>+ content = new String(b).trim();
>+ bytes = null;
>+ } catch (Exception e) {
>+ //TODO log an exception here
>+ //throw FaultCodes.createXMLDBException(e);
>+ }
>+
>+ }
>+
>+ };
>+
>+ return new BufferedOutputStream(pipedOut, BUFFER_SIZE);
> }
>
> /**
>
>