You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ibatis.apache.org by "Daigo Kobayashi (JIRA)" <ib...@incubator.apache.org> on 2006/09/26 03:06:50 UTC
[jira] Created: (IBATIS-349) NodeletParser ignore XML encoding.
NodeletParser ignore XML encoding.
----------------------------------
Key: IBATIS-349
URL: http://issues.apache.org/jira/browse/IBATIS-349
Project: iBatis for Java
Issue Type: Bug
Components: SQL Maps
Affects Versions: 2.2.0
Environment: Windows XP(Japanese)
jdk 1.5.0_06
Reporter: Daigo Kobayashi
NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Jeff Butler (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12453670 ]
Jeff Butler commented on IBATIS-349:
------------------------------------
Would it work for you if we added methods to Resources like this:
Resources.getResourceAsReader(String resource, String encoding) {
return new InputStreamReader(getResourceAsStream(resource), Charset.forName(encoding));
}
Resources.getResourceAsReader(ClassLoader loader, String resource, String encoding) {
return new InputStreamReader(getResourceAsStream(loader, resource), Charset.forName(encoding));
}
This would be a lot simpler than opening the XML file twice - the first time just to get the encoding.
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Daigo Kobayashi (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12454512 ]
Daigo Kobayashi commented on IBATIS-349:
----------------------------------------
I posted enhancement request .
http://opensource.atlassian.com/projects/spring/browse/SPR-2895
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Fix For: 2.3.0
>
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Daigo Kobayashi (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12454246 ]
Daigo Kobayashi commented on IBATIS-349:
----------------------------------------
This works fine. And this is ideal solution.
I really appriciate your great job.
BTW, I'm using iBatis with springframework. So I have to fix spring's SqlMapClientFactoryBean.java code. (trivial change.)
So I think some action to spring team is necessary.
Could you report this problem to spring team? Or should I report?
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Daigo Kobayashi (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12448161 ]
Daigo Kobayashi commented on IBATIS-349:
----------------------------------------
Is there any progress? Attachment is not enough?
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Jeff Butler (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12453773 ]
Jeff Butler commented on IBATIS-349:
------------------------------------
Well, my first comment works for the SqlMapConfig file, but doesn't provide any help for the SqlMap files.
Take a look at IBATIS-340 - a similar problem. Of the two solutions, I like the solution proposed in IBATIS-340 better. That's what I'd like to do unless you have some compelling reason that it won't work.
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Daigo Kobayashi (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12447338 ]
Daigo Kobayashi commented on IBATIS-349:
----------------------------------------
Is it possible to merge attatched code before next release 2.2.1 (or 2.3) ?
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Jeff Butler (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12454155 ]
Jeff Butler commented on IBATIS-349:
------------------------------------
I've committed some changes for IBATIS-373 that allow iBATIS to build the SqlMapClient from an InputStream rather than a Reader. This delegates the encoding issue to the parser - where it should be IMHO. You can use it like this:
String resource = "myconfig/SqlMapConfig.xml";
InputStream inputStream = Resources.getResourceAsStream(resource);
SqlMapClient client = SqlMapClientBuilder.buildSqlMapClient(inputStream);
Please give this a try (or you can wait for 2.3.0 later this week). If one of these alternatives resolves the issue, then I'd like to close this ticket.
Thanks for all the feedback!
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Jeff Butler (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12454403 ]
Jeff Butler commented on IBATIS-349:
------------------------------------
Great! I'm glad this fix worked.
I think you should open the enhancement request with the Spring team. You could probably provide your modification directly to them, and you're in a better position to test than I am.
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Daigo Kobayashi (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12453970 ]
Daigo Kobayashi commented on IBATIS-349:
----------------------------------------
XML file have it's own encoding. So we should not ignore that encoding. If we follow this solution, we have to set encoding and ignore xml's encoding.
And xml specification says, if encoding is not specified parser must use UTF-8 or UTF-16 and if specified follow it. So this solution violate xml's encoding specification.
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Jeff Butler (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=all ]
Jeff Butler closed IBATIS-349.
------------------------------
Fix Version/s: 2.3.0
Resolution: Fixed
This issue is resolved with the changes for IBATIS-373.
See comments for more details.
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Fix For: 2.3.0
>
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Jeff Butler (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12454071 ]
Jeff Butler commented on IBATIS-349:
------------------------------------
This is a complex issue. The real problem is that iBATIS ignores the encoding issue altogether in parsing - because we only accept a character reader.
I had a quick look at Xerces source to see how they autodetect the encoding and their method is quite different than what you propose. Your method still uses the system default encoding to open the file, a potential source of errors. And it ignores the fact that the real encoding of the file may be different than what's declared in the processing instruction. Xerces figures out the encoding by looking at the first 4 bytes of the file. This is a complex algorithm that we don't want to replicate in iBATIS.
I think the best way to resolve this is to allow iBATIS to accept a byte stream for parsing - then we could leverage the parser's built in support for dealing with different encodings. I'll have a look at doing that.
In the meantime, please try manually setting the encoding to UTF-8 using the fix for IBATIS-340. I'd like to know if this addresses your immediate issue.
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.java, Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (IBATIS-349) NodeletParser ignore XML encoding.
Posted by "Clinton Begin (JIRA)" <ib...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/IBATIS-349?page=comments#action_12447345 ]
Clinton Begin commented on IBATIS-349:
--------------------------------------
Can you upload the whole file? Not just the patch. Believe it or not, it's much easier to deal with.
> NodeletParser ignore XML encoding.
> ----------------------------------
>
> Key: IBATIS-349
> URL: http://issues.apache.org/jira/browse/IBATIS-349
> Project: iBatis for Java
> Issue Type: Bug
> Components: SQL Maps
> Affects Versions: 2.2.0
> Environment: Windows XP(Japanese)
> jdk 1.5.0_06
> Reporter: Daigo Kobayashi
> Attachments: Resources.patch, Resources.patch
>
>
> NodeletParser ignore XML encoding. Because NodeletParser use Resources, and in some case Resources use ClassLoader#getResourceAsStream with OS default encoding not XML's.
> In some environment, for example OS use Shift_JIS and XML use UTF-8, iBatis doesn't work correctly because of broken character.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira