You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Ryan McKinley (JIRA)" <ji...@apache.org> on 2007/05/09 19:54:15 UTC
[jira] Created: (SOLR-231) By default, use UTF-8 for posted content
streams
By default, use UTF-8 for posted content streams
------------------------------------------------
Key: SOLR-231
URL: https://issues.apache.org/jira/browse/SOLR-231
Project: Solr
Issue Type: Improvement
Reporter: Ryan McKinley
Fix For: 1.2
Solr should assume UTF-8 encoding unless the contentType says otherwise. To change the contentType and encoding set the header value with contentType ="text/xml; charset=utf-8"
likewise, with stream.body=xxxx, will default to UTF-8 unless the stream.contentType says otherwise.
For previous discussion, see:
http://www.nabble.com/resin-and-UTF-8-in-URLs-tf3152910.html
http://www.nabble.com/charset-in-POST-from-browser-tf3153057.html
http://www.nabble.com/Re%3A-svn-commit%3A-r536048----lucene-solr-trunk-src-webapp-src-org-apache-solr-servlet-SolrRequestParsers.java-tf3712816.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-231) By default, use UTF-8 for posted
content streams
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494556 ]
Hoss Man commented on SOLR-231:
-------------------------------
Yonik: agreed that the XML parsing should (eventually) use the raw InputStream instead of a Reader if no explicit charset is declaured in teh content type ... but that's a seperate issue (SOLR-96) specific to XmlUpdateRequestHandler.
Independent of that is the question: "what should an arbitrary request handler get if it calls ContentStream.getReader and the ContentStream doesn't know explicitly know the charset of the InputStream it has?"
The patch seems clean to me.
> By default, use UTF-8 for posted content streams
> ------------------------------------------------
>
> Key: SOLR-231
> URL: https://issues.apache.org/jira/browse/SOLR-231
> Project: Solr
> Issue Type: Improvement
> Reporter: Ryan McKinley
> Fix For: 1.2
>
> Attachments: SOLR-231-ContentType-UTF8.patch, SOLR-231-ContentType-UTF8.patch
>
>
> Solr should assume UTF-8 encoding unless the contentType says otherwise. To change the contentType and encoding set the header value with contentType ="text/xml; charset=utf-8"
> likewise, with stream.body=xxxx, will default to UTF-8 unless the stream.contentType says otherwise.
>
> For previous discussion, see:
> http://www.nabble.com/resin-and-UTF-8-in-URLs-tf3152910.html
> http://www.nabble.com/charset-in-POST-from-browser-tf3153057.html
> http://www.nabble.com/Re%3A-svn-commit%3A-r536048----lucene-solr-trunk-src-webapp-src-org-apache-solr-servlet-SolrRequestParsers.java-tf3712816.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-231) By default, use UTF-8 for posted content
streams
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley updated SOLR-231:
-------------------------------
Attachment: SOLR-231-ContentType-UTF8.patch
oops, need to grant license
> By default, use UTF-8 for posted content streams
> ------------------------------------------------
>
> Key: SOLR-231
> URL: https://issues.apache.org/jira/browse/SOLR-231
> Project: Solr
> Issue Type: Improvement
> Reporter: Ryan McKinley
> Fix For: 1.2
>
> Attachments: SOLR-231-ContentType-UTF8.patch, SOLR-231-ContentType-UTF8.patch
>
>
> Solr should assume UTF-8 encoding unless the contentType says otherwise. To change the contentType and encoding set the header value with contentType ="text/xml; charset=utf-8"
> likewise, with stream.body=xxxx, will default to UTF-8 unless the stream.contentType says otherwise.
>
> For previous discussion, see:
> http://www.nabble.com/resin-and-UTF-8-in-URLs-tf3152910.html
> http://www.nabble.com/charset-in-POST-from-browser-tf3153057.html
> http://www.nabble.com/Re%3A-svn-commit%3A-r536048----lucene-solr-trunk-src-webapp-src-org-apache-solr-servlet-SolrRequestParsers.java-tf3712816.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-231) By default, use UTF-8 for posted
content streams
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494554 ]
Ryan McKinley commented on SOLR-231:
------------------------------------
>> Solr should assume UTF-8 encoding unless the contentType says otherwise.
>
> In general yes (when Solr is asked for a Reader).
> For XML, we should probably give the parser an InputStream.
> http://www.nabble.com/double-curl-calls-in-post.sh--tf2287469.html#a6369448
>
sounds good. This patch only affects what charset is used when you call getReader()
Perhaps as part of SOLR-133, we should make sure it it passed uses the getInputStream() method.
> By default, use UTF-8 for posted content streams
> ------------------------------------------------
>
> Key: SOLR-231
> URL: https://issues.apache.org/jira/browse/SOLR-231
> Project: Solr
> Issue Type: Improvement
> Reporter: Ryan McKinley
> Fix For: 1.2
>
> Attachments: SOLR-231-ContentType-UTF8.patch, SOLR-231-ContentType-UTF8.patch
>
>
> Solr should assume UTF-8 encoding unless the contentType says otherwise. To change the contentType and encoding set the header value with contentType ="text/xml; charset=utf-8"
> likewise, with stream.body=xxxx, will default to UTF-8 unless the stream.contentType says otherwise.
>
> For previous discussion, see:
> http://www.nabble.com/resin-and-UTF-8-in-URLs-tf3152910.html
> http://www.nabble.com/charset-in-POST-from-browser-tf3153057.html
> http://www.nabble.com/Re%3A-svn-commit%3A-r536048----lucene-solr-trunk-src-webapp-src-org-apache-solr-servlet-SolrRequestParsers.java-tf3712816.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-231) By default, use UTF-8 for posted
content streams
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley resolved SOLR-231.
--------------------------------
Resolution: Fixed
added in 537024
> By default, use UTF-8 for posted content streams
> ------------------------------------------------
>
> Key: SOLR-231
> URL: https://issues.apache.org/jira/browse/SOLR-231
> Project: Solr
> Issue Type: Improvement
> Reporter: Ryan McKinley
> Assigned To: Ryan McKinley
> Fix For: 1.2
>
> Attachments: SOLR-231-ContentType-UTF8.patch, SOLR-231-ContentType-UTF8.patch
>
>
> Solr should assume UTF-8 encoding unless the contentType says otherwise. To change the contentType and encoding set the header value with contentType ="text/xml; charset=utf-8"
> likewise, with stream.body=xxxx, will default to UTF-8 unless the stream.contentType says otherwise.
>
> For previous discussion, see:
> http://www.nabble.com/resin-and-UTF-8-in-URLs-tf3152910.html
> http://www.nabble.com/charset-in-POST-from-browser-tf3153057.html
> http://www.nabble.com/Re%3A-svn-commit%3A-r536048----lucene-solr-trunk-src-webapp-src-org-apache-solr-servlet-SolrRequestParsers.java-tf3712816.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-231) By default, use UTF-8 for posted
content streams
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley reassigned SOLR-231:
----------------------------------
Assignee: Ryan McKinley
> By default, use UTF-8 for posted content streams
> ------------------------------------------------
>
> Key: SOLR-231
> URL: https://issues.apache.org/jira/browse/SOLR-231
> Project: Solr
> Issue Type: Improvement
> Reporter: Ryan McKinley
> Assigned To: Ryan McKinley
> Fix For: 1.2
>
> Attachments: SOLR-231-ContentType-UTF8.patch, SOLR-231-ContentType-UTF8.patch
>
>
> Solr should assume UTF-8 encoding unless the contentType says otherwise. To change the contentType and encoding set the header value with contentType ="text/xml; charset=utf-8"
> likewise, with stream.body=xxxx, will default to UTF-8 unless the stream.contentType says otherwise.
>
> For previous discussion, see:
> http://www.nabble.com/resin-and-UTF-8-in-URLs-tf3152910.html
> http://www.nabble.com/charset-in-POST-from-browser-tf3153057.html
> http://www.nabble.com/Re%3A-svn-commit%3A-r536048----lucene-solr-trunk-src-webapp-src-org-apache-solr-servlet-SolrRequestParsers.java-tf3712816.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-231) By default, use UTF-8 for posted
content streams
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12494551 ]
Yonik Seeley commented on SOLR-231:
-----------------------------------
> Solr should assume UTF-8 encoding unless the contentType says otherwise.
In general yes (when Solr is asked for a Reader).
For XML, we should probably give the parser an InputStream.
http://www.nabble.com/double-curl-calls-in-post.sh--tf2287469.html#a6369448
> By default, use UTF-8 for posted content streams
> ------------------------------------------------
>
> Key: SOLR-231
> URL: https://issues.apache.org/jira/browse/SOLR-231
> Project: Solr
> Issue Type: Improvement
> Reporter: Ryan McKinley
> Fix For: 1.2
>
> Attachments: SOLR-231-ContentType-UTF8.patch, SOLR-231-ContentType-UTF8.patch
>
>
> Solr should assume UTF-8 encoding unless the contentType says otherwise. To change the contentType and encoding set the header value with contentType ="text/xml; charset=utf-8"
> likewise, with stream.body=xxxx, will default to UTF-8 unless the stream.contentType says otherwise.
>
> For previous discussion, see:
> http://www.nabble.com/resin-and-UTF-8-in-URLs-tf3152910.html
> http://www.nabble.com/charset-in-POST-from-browser-tf3153057.html
> http://www.nabble.com/Re%3A-svn-commit%3A-r536048----lucene-solr-trunk-src-webapp-src-org-apache-solr-servlet-SolrRequestParsers.java-tf3712816.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-231) By default, use UTF-8 for posted content
streams
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley updated SOLR-231:
-------------------------------
Attachment: SOLR-231-ContentType-UTF8.patch
This patch also makes sure the behavior is consistent for multipart file uploads
> By default, use UTF-8 for posted content streams
> ------------------------------------------------
>
> Key: SOLR-231
> URL: https://issues.apache.org/jira/browse/SOLR-231
> Project: Solr
> Issue Type: Improvement
> Reporter: Ryan McKinley
> Fix For: 1.2
>
> Attachments: SOLR-231-ContentType-UTF8.patch
>
>
> Solr should assume UTF-8 encoding unless the contentType says otherwise. To change the contentType and encoding set the header value with contentType ="text/xml; charset=utf-8"
> likewise, with stream.body=xxxx, will default to UTF-8 unless the stream.contentType says otherwise.
>
> For previous discussion, see:
> http://www.nabble.com/resin-and-UTF-8-in-URLs-tf3152910.html
> http://www.nabble.com/charset-in-POST-from-browser-tf3153057.html
> http://www.nabble.com/Re%3A-svn-commit%3A-r536048----lucene-solr-trunk-src-webapp-src-org-apache-solr-servlet-SolrRequestParsers.java-tf3712816.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.