You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by "John Snelson (JIRA)" <xe...@xml.apache.org> on 2008/05/15 17:13:55 UTC
[jira] Created: (XERCESC-1805) Accessing the HTTP Content-Type
Accessing the HTTP Content-Type
-------------------------------
Key: XERCESC-1805
URL: https://issues.apache.org/jira/browse/XERCESC-1805
Project: Xerces-C++
Issue Type: Improvement
Components: Miscellaneous
Affects Versions: 3.0.0
Reporter: John Snelson
Fix For: 3.0.0
A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
http://www.w3.org/TR/xslt20/#unparsed-text
We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
1) They used fixed buffers which can result in buffer overflow.
2) They needlessly duplicate a whole load of code that could be shared.
3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1805) Accessing the HTTP Content-Type
Posted by "John Snelson (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12606334#action_12606334 ]
John Snelson commented on XERCESC-1805:
---------------------------------------
Hi Boris,
I believe there are file systems that support mime-types, as well as version control systems and other means of getting files. For this reason it seems wrong to restrict the getContentType() method to only an HTTP input stream. I also would be very reluctant to add anything to the API that requires up-casting, especially using dynamic_cast.
(I know, I know, W3C DOM requires up-casting - but that doesn't make it good design... )
Given that the format of the "Content-Type" HTTP header is more or less a defacto standard for representing mime-type and encoding, maybe we can just specify in the documentation that that will be the format of the string returned?
John
> Accessing the HTTP Content-Type
> -------------------------------
>
> Key: XERCESC-1805
> URL: https://issues.apache.org/jira/browse/XERCESC-1805
> Project: Xerces-C++
> Issue Type: Improvement
> Components: Miscellaneous
> Affects Versions: 3.0.0
> Reporter: John Snelson
> Fix For: 3.0.0
>
> Attachments: xercesc_3_0_content_type_patch
>
>
> A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
> http://www.w3.org/TR/xslt20/#unparsed-text
> We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
> In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
> 1) They used fixed buffers which can result in buffer overflow.
> 2) They needlessly duplicate a whole load of code that could be shared.
> 3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Closed: (XERCESC-1805) Accessing the HTTP Content-Type
Posted by "Boris Kolpackov (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Kolpackov closed XERCESC-1805.
------------------------------------
Resolution: Fixed
Assignee: Boris Kolpackov
The pacth is in SVN, thank you. I think there are no known problems left so I am closing this issue.
> Accessing the HTTP Content-Type
> -------------------------------
>
> Key: XERCESC-1805
> URL: https://issues.apache.org/jira/browse/XERCESC-1805
> Project: Xerces-C++
> Issue Type: Improvement
> Components: Miscellaneous
> Affects Versions: 3.0.0
> Reporter: John Snelson
> Assignee: Boris Kolpackov
> Fix For: 3.0.0
>
> Attachments: curl_and_local_code_page_patch, xercesc_3_0_content_type_patch
>
>
> A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
> http://www.w3.org/TR/xslt20/#unparsed-text
> We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
> In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
> 1) They used fixed buffers which can result in buffer overflow.
> 2) They needlessly duplicate a whole load of code that could be shared.
> 3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1805) Accessing the HTTP Content-Type
Posted by "Boris Kolpackov (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12606726#action_12606726 ]
Boris Kolpackov commented on XERCESC-1805:
------------------------------------------
Hi Jonh,
I don't know, it all feels wrong. Let's assume we specify that getContentType() returns "media-type" as defined in the HTTP 1.1 spec. The next issue is when can getContentType() be called? I would assume most users would want to know the content type before actually reading any data. But this does not seem to be valid for HTTP: getContentType can only be called after readBytes (before that the request hasn't been made so the content type is not known; I wonder how do you even use something like this in XQilla...). So I feel uncomfotable putting getContentType which appears to be very closely tied to the HTTP stream into the base interface. Any thoughts on this?
Another issue that I noticed is that you use XMLString::transcode on the value of Content-Type which assumes that it is in local code page. But this is minor so perhaps you can submit a small patch later once I commit the major changes.
Boris
> Accessing the HTTP Content-Type
> -------------------------------
>
> Key: XERCESC-1805
> URL: https://issues.apache.org/jira/browse/XERCESC-1805
> Project: Xerces-C++
> Issue Type: Improvement
> Components: Miscellaneous
> Affects Versions: 3.0.0
> Reporter: John Snelson
> Fix For: 3.0.0
>
> Attachments: xercesc_3_0_content_type_patch
>
>
> A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
> http://www.w3.org/TR/xslt20/#unparsed-text
> We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
> In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
> 1) They used fixed buffers which can result in buffer overflow.
> 2) They needlessly duplicate a whole load of code that could be shared.
> 3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1805) Accessing the HTTP Content-Type
Posted by "Boris Kolpackov (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607094#action_12607094 ]
Boris Kolpackov commented on XERCESC-1805:
------------------------------------------
Hi John,
Ok, I've commited a slightly-modified version of your changes (made getContentType pure virtual, moved the common implementation to NetAccessors/).
If you can provide a patch that fixes the following then I will be happy to review & commit it:
1. Don't use XMLString::transcode on the value of Content-Type
2. Make sure getContentType can be called after construction in Curl
While at it, perahps you could also fix up Curl implementation not to use local code page like you did for Sockect and Winsock. Curl is a default net accessors (if libcurl is available) on all UNIX/Linux platforms so it would be good to fix this.
Thanks,
Boris
> Accessing the HTTP Content-Type
> -------------------------------
>
> Key: XERCESC-1805
> URL: https://issues.apache.org/jira/browse/XERCESC-1805
> Project: Xerces-C++
> Issue Type: Improvement
> Components: Miscellaneous
> Affects Versions: 3.0.0
> Reporter: John Snelson
> Fix For: 3.0.0
>
> Attachments: xercesc_3_0_content_type_patch
>
>
> A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
> http://www.w3.org/TR/xslt20/#unparsed-text
> We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
> In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
> 1) They used fixed buffers which can result in buffer overflow.
> 2) They needlessly duplicate a whole load of code that could be shared.
> 3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Updated: (XERCESC-1805) Accessing the HTTP Content-Type
Posted by "John Snelson (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Snelson updated XERCESC-1805:
----------------------------------
Attachment: xercesc_3_0_content_type_patch
This patch implements the BinInputStream::getContentType() method for the socket, winsock and libcurl net accessors.
In addition, the common code has been extracted from the UnixHTTPURLInputStream (socket) and BinHTTPURLInputStream (winsock), cleaned up, bug fixed and put into two new files -
src/xercesc/util/BinHTTPInputStream.hpp
src/xercesc/util/BinHTTPInputStream.cpp
These files have been added to src/Makefile.am, but not to any project files.
This patch has been tested on Ubuntu using libcurl and socket net accessors, and on Windows XP MingW using winsock.
> Accessing the HTTP Content-Type
> -------------------------------
>
> Key: XERCESC-1805
> URL: https://issues.apache.org/jira/browse/XERCESC-1805
> Project: Xerces-C++
> Issue Type: Improvement
> Components: Miscellaneous
> Affects Versions: 3.0.0
> Reporter: John Snelson
> Fix For: 3.0.0
>
> Attachments: xercesc_3_0_content_type_patch
>
>
> A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
> http://www.w3.org/TR/xslt20/#unparsed-text
> We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
> In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
> 1) They used fixed buffers which can result in buffer overflow.
> 2) They needlessly duplicate a whole load of code that could be shared.
> 3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Updated: (XERCESC-1805) Accessing the HTTP Content-Type
Posted by "John Snelson (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Snelson updated XERCESC-1805:
----------------------------------
Attachment: curl_and_local_code_page_patch
This patch fixes CurlURLInputStream::getContentType(), and changes incorrect transcoding to and from the local codepage to use iso-8859-1.
As an added bonus it also implements redirects, authentication, PUT, POST, custom headers and payloads in the CurlURLInputStream.
> Accessing the HTTP Content-Type
> -------------------------------
>
> Key: XERCESC-1805
> URL: https://issues.apache.org/jira/browse/XERCESC-1805
> Project: Xerces-C++
> Issue Type: Improvement
> Components: Miscellaneous
> Affects Versions: 3.0.0
> Reporter: John Snelson
> Fix For: 3.0.0
>
> Attachments: curl_and_local_code_page_patch, xercesc_3_0_content_type_patch
>
>
> A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
> http://www.w3.org/TR/xslt20/#unparsed-text
> We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
> In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
> 1) They used fixed buffers which can result in buffer overflow.
> 2) They needlessly duplicate a whole load of code that could be shared.
> 3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1805) Accessing the HTTP Content-Type
Posted by "John Snelson (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12606799#action_12606799 ]
John Snelson commented on XERCESC-1805:
---------------------------------------
Hi Boris,
The getContentType() method can be called after construction of the relevant BinInputStream implementations, since the HTTP headers are read during construction.
I fixed a number of transcoding bugs in the original HTTP BinInputStream implementations, so I must have missed the one you're talking about. Another patch to fix that issue should be simple to add.
John
> Accessing the HTTP Content-Type
> -------------------------------
>
> Key: XERCESC-1805
> URL: https://issues.apache.org/jira/browse/XERCESC-1805
> Project: Xerces-C++
> Issue Type: Improvement
> Components: Miscellaneous
> Affects Versions: 3.0.0
> Reporter: John Snelson
> Fix For: 3.0.0
>
> Attachments: xercesc_3_0_content_type_patch
>
>
> A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
> http://www.w3.org/TR/xslt20/#unparsed-text
> We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
> In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
> 1) They used fixed buffers which can result in buffer overflow.
> 2) They needlessly duplicate a whole load of code that could be shared.
> 3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] Commented: (XERCESC-1805) Accessing the HTTP Content-Type
Posted by "Boris Kolpackov (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12606306#action_12606306 ]
Boris Kolpackov commented on XERCESC-1805:
------------------------------------------
Hi John,
I reviewed your patch and it looks good overall. I have one suggestion,
however. It is about the getContentType() function. Its documentation
says that it returns a content type in some unspecified form. For
HTTP it it the value of the Content-Type header. For other stream types
it can be something else. As a result, I don't see how this function
can be used unless the user knows what to expect from it (that is,
the user knows the stream is HTTP and the getContentType() will
return the Content-Type header).
This made me think that perhaps placing getContentType() in the
BinInputStream interface is not a best choice. Perhaps it would
be better to create the BinHTTPInputStream interface which extends
the BinInputStream and adds a (pure virtual) getContentType(). Then
all HTTP stream implementations will derive from BinHTTPInputStream.
We will also change the makeNew() function in XMLNetAccessor to return
BinHTTPInputStream instead of BinInputStream.
Even if the user has a BinInputStream instead of BinHTTPInputStream,
he can always try to dynamic_cast to BinHTTPInputStream to see if the
content type is available. In fact I think this is the only way for him
to know what to expect from getContentType().
So here is how I suggest we change the current patch: Create
src/xercesc/util/BinHTTPInputStream.hpp with the BinHTTPInputStream
interface. This accidently conflicts with the common implementation
that you have created for Winsock and Socket. We can rename those
files to
BinHTTPInputStreamCommon.hpp
BinHTTPInputStreamCommon.cpp
and move them to the NetAccessor directory (they, BTW, only need
to be compiled when Socket or Winsock accessors are used so it
would make sense to move them to NetAccessor).
Let me know if you are ok with this and I will go ahead and apply
the patch and make the changes.
Boris
> Accessing the HTTP Content-Type
> -------------------------------
>
> Key: XERCESC-1805
> URL: https://issues.apache.org/jira/browse/XERCESC-1805
> Project: Xerces-C++
> Issue Type: Improvement
> Components: Miscellaneous
> Affects Versions: 3.0.0
> Reporter: John Snelson
> Fix For: 3.0.0
>
> Attachments: xercesc_3_0_content_type_patch
>
>
> A lot of algorithms need access to the HTTP "Content-Type" header, to decide how to parse a file, or what encoding it is in - for instance see XSLT 2.0's unparsed-text() function:
> http://www.w3.org/TR/xslt20/#unparsed-text
> We should add a method, BinInputStream::getContentType(), and implement it in the HTTP input stream implementations. The method should return 0 when the content type is not available, like for file input streams.
> In addition, the socket and WinSock HTTP InputStream implementations have a number of problems:
> 1) They used fixed buffers which can result in buffer overflow.
> 2) They needlessly duplicate a whole load of code that could be shared.
> 3) They transcode to the local code page rather than "ISO8859-1".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org