You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@shale.apache.org by "Tom Pasierb (JIRA)" <ji...@apache.org> on 2006/09/25 19:04:31 UTC
[jira] Created: (SHALE-292) Clay doesn't consider file's encoding
when loading/parsing html templates from hdd
Clay doesn't consider file's encoding when loading/parsing html templates from hdd
----------------------------------------------------------------------------------
Key: SHALE-292
URL: http://issues.apache.org/struts/browse/SHALE-292
Project: Shale
Issue Type: Bug
Components: Clay
Affects Versions: 1.0.3, 1.0.2, 1.0.1, 1.0.0, 1.0.4-SNAPSHOT
Environment: windows xp, tomcat 5.5 (started with -Dfile.encoding=UTF-8 option, this way myfaces doesn't convert all non-ascii characters to html entities), myfaces 1.1.3
Reporter: Tom Pasierb
Clay reads html files assuming ascii encoding. This way it's impossible to have characters other than ascii in templates. They do not display correctly. As indicated on the user mailing list a Reader object should be used for reading templates instead of InputStream. I wrote more about this on shale user mailing list.
We probably need:
1. app wide config option for setting encoding clay should use for reading templates in. Clay would default to this setting unless maybe
2. some per file encoding config option was set (something similar to @page pageEncoding directive for jsps)
I marked it as major as this should be corrected if one wants to develop localized applications with non-ascii characters in html templates.
As noted by Craig this probably also applies to xml templates, which I haven't tried myself.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/struts/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (SHALE-292) Clay doesn't consider file's encoding
when loading/parsing html templates from hdd
Posted by "Gary VanMatre (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/struts/browse/SHALE-292?page=all ]
Gary VanMatre reassigned SHALE-292:
-----------------------------------
Assignee: Gary VanMatre
> Clay doesn't consider file's encoding when loading/parsing html templates from hdd
> ----------------------------------------------------------------------------------
>
> Key: SHALE-292
> URL: http://issues.apache.org/struts/browse/SHALE-292
> Project: Shale
> Issue Type: Bug
> Components: Clay
> Affects Versions: 1.0.0, 1.0.1, 1.0.2, 1.0.4-SNAPSHOT, 1.0.3
> Environment: windows xp, tomcat 5.5 (started with -Dfile.encoding=UTF-8 option, this way myfaces doesn't convert all non-ascii characters to html entities), myfaces 1.1.3
> Reporter: Tom Pasierb
> Assigned To: Gary VanMatre
> Attachments: some.html, whatever.jsp
>
>
> Clay reads html files assuming ascii encoding. This way it's impossible to have characters other than ascii in templates. They do not display correctly. As indicated on the user mailing list a Reader object should be used for reading templates instead of InputStream. I wrote more about this on shale user mailing list.
> We probably need:
> 1. app wide config option for setting encoding clay should use for reading templates in. Clay would default to this setting unless maybe
> 2. some per file encoding config option was set (something similar to @page pageEncoding directive for jsps)
> I marked it as major as this should be corrected if one wants to develop localized applications with non-ascii characters in html templates.
> As noted by Craig this probably also applies to xml templates, which I haven't tried myself.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/struts/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SHALE-292) Clay doesn't consider file's encoding
when loading/parsing html templates from hdd
Posted by "Gary VanMatre (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/struts/browse/SHALE-292?page=comments#action_38283 ]
Gary VanMatre commented on SHALE-292:
-------------------------------------
This is the first try at resolving this issue. It will be available in the shale-framework-20060928 nightly build. You can find it here: http://people.apache.org/builds/shale/nightly/.
To summarize the changes based on your notes the encoding is now determined with the following steps:
1) Look at the top of the template for the token comment containing the charset.
For example: <-- ### clay:page charset="UTF-8" /### -->
2) If not found, look for the app wide config option for template encoding. If found use the encoding for reading the template.
For example:
<context-param>
<param-name>org.apache.shale.clay.HTML_TEMPLATE_CHARSET</param-name>
<param-value>UTF-8</param-value>
</context-param>
3) If not found use the vm's default "file.encoding".
4) Read the template in with the determined encoding
Tom, I'm going to leave this open until you have a chance to verify.
> Clay doesn't consider file's encoding when loading/parsing html templates from hdd
> ----------------------------------------------------------------------------------
>
> Key: SHALE-292
> URL: http://issues.apache.org/struts/browse/SHALE-292
> Project: Shale
> Issue Type: Bug
> Components: Clay
> Affects Versions: 1.0.0, 1.0.1, 1.0.2, 1.0.4-SNAPSHOT, 1.0.3
> Environment: windows xp, tomcat 5.5 (started with -Dfile.encoding=UTF-8 option, this way myfaces doesn't convert all non-ascii characters to html entities), myfaces 1.1.3
> Reporter: Tom Pasierb
> Assigned To: Gary VanMatre
> Attachments: some.html, whatever.jsp
>
>
> Clay reads html files assuming ascii encoding. This way it's impossible to have characters other than ascii in templates. They do not display correctly. As indicated on the user mailing list a Reader object should be used for reading templates instead of InputStream. I wrote more about this on shale user mailing list.
> We probably need:
> 1. app wide config option for setting encoding clay should use for reading templates in. Clay would default to this setting unless maybe
> 2. some per file encoding config option was set (something similar to @page pageEncoding directive for jsps)
> I marked it as major as this should be corrected if one wants to develop localized applications with non-ascii characters in html templates.
> As noted by Craig this probably also applies to xml templates, which I haven't tried myself.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/struts/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (SHALE-292) Clay doesn't consider file's encoding
when loading/parsing html templates from hdd
Posted by "Gary VanMatre (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/struts/browse/SHALE-292?page=all ]
Gary VanMatre resolved SHALE-292.
---------------------------------
Fix Version/s: 1.0.4-SNAPSHOT
Resolution: Fixed
The examples and your input really made the difference on this one. Thanks for the help Tom.
> Clay doesn't consider file's encoding when loading/parsing html templates from hdd
> ----------------------------------------------------------------------------------
>
> Key: SHALE-292
> URL: http://issues.apache.org/struts/browse/SHALE-292
> Project: Shale
> Issue Type: Bug
> Components: Clay
> Affects Versions: 1.0.0, 1.0.1, 1.0.2, 1.0.4-SNAPSHOT, 1.0.3
> Environment: windows xp, tomcat 5.5 (started with -Dfile.encoding=UTF-8 option, this way myfaces doesn't convert all non-ascii characters to html entities), myfaces 1.1.3
> Reporter: Tom Pasierb
> Assigned To: Gary VanMatre
> Fix For: 1.0.4-SNAPSHOT
>
> Attachments: some.html, whatever.jsp
>
>
> Clay reads html files assuming ascii encoding. This way it's impossible to have characters other than ascii in templates. They do not display correctly. As indicated on the user mailing list a Reader object should be used for reading templates instead of InputStream. I wrote more about this on shale user mailing list.
> We probably need:
> 1. app wide config option for setting encoding clay should use for reading templates in. Clay would default to this setting unless maybe
> 2. some per file encoding config option was set (something similar to @page pageEncoding directive for jsps)
> I marked it as major as this should be corrected if one wants to develop localized applications with non-ascii characters in html templates.
> As noted by Craig this probably also applies to xml templates, which I haven't tried myself.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/struts/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SHALE-292) Clay doesn't consider file's encoding
when loading/parsing html templates from hdd
Posted by "Tom Pasierb (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/struts/browse/SHALE-292?page=comments#action_38313 ]
Tom Pasierb commented on SHALE-292:
-----------------------------------
I have tried the updated clay version with html templates.
I experimented with -Dfile.encoding (system encoding setting), org.apache.shale.clay.HTML_TEMPLATE_CHARSET context init parameter and <-- ### clay:page charset="UTF-8" /### --> and everything works as expected so I guess this issue can be closed.
Thanks Gary :-)
> Clay doesn't consider file's encoding when loading/parsing html templates from hdd
> ----------------------------------------------------------------------------------
>
> Key: SHALE-292
> URL: http://issues.apache.org/struts/browse/SHALE-292
> Project: Shale
> Issue Type: Bug
> Components: Clay
> Affects Versions: 1.0.0, 1.0.1, 1.0.2, 1.0.4-SNAPSHOT, 1.0.3
> Environment: windows xp, tomcat 5.5 (started with -Dfile.encoding=UTF-8 option, this way myfaces doesn't convert all non-ascii characters to html entities), myfaces 1.1.3
> Reporter: Tom Pasierb
> Assigned To: Gary VanMatre
> Attachments: some.html, whatever.jsp
>
>
> Clay reads html files assuming ascii encoding. This way it's impossible to have characters other than ascii in templates. They do not display correctly. As indicated on the user mailing list a Reader object should be used for reading templates instead of InputStream. I wrote more about this on shale user mailing list.
> We probably need:
> 1. app wide config option for setting encoding clay should use for reading templates in. Clay would default to this setting unless maybe
> 2. some per file encoding config option was set (something similar to @page pageEncoding directive for jsps)
> I marked it as major as this should be corrected if one wants to develop localized applications with non-ascii characters in html templates.
> As noted by Craig this probably also applies to xml templates, which I haven't tried myself.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/struts/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SHALE-292) Clay doesn't consider file's encoding
when loading/parsing html templates from hdd
Posted by "Gary VanMatre (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/struts/browse/SHALE-292?page=comments#action_38273 ]
Gary VanMatre commented on SHALE-292:
-------------------------------------
I like the idea of being able to specify the encoding per document template. Many templates can be included in a single page. We might be able to build on a similar idea that is built into the clay markup parser.
The clay HTML template parser has a couple special tokens that it used to block out a markup that should be excluded form the document. These tokens are in the form of comments.
<!-- ### clay:remove ### -->
<html>
exclude this text
<!-- ### /clay:remove ### -->
These special comments and the markup between is ignored - dropped from the document.
What if we used another special comment token that operates like a page directive?
<!-- ### clay:page charset="UTF-8" / -->
This comment would have to be in the first few bytes of the template document. The top of each template could be sniffed for this token. If it exists, extract the charset and open the target template with the specified encoding.
Reading each template file would be broken down into two steps.
1) Look at the top of the template for the token comment containing the charset. If not found, use the vm's default "file.encoding".
2) Read the template in with the determined encoding
Does is this a sound plan? Any thoughts?
> Clay doesn't consider file's encoding when loading/parsing html templates from hdd
> ----------------------------------------------------------------------------------
>
> Key: SHALE-292
> URL: http://issues.apache.org/struts/browse/SHALE-292
> Project: Shale
> Issue Type: Bug
> Components: Clay
> Affects Versions: 1.0.0, 1.0.1, 1.0.2, 1.0.4-SNAPSHOT, 1.0.3
> Environment: windows xp, tomcat 5.5 (started with -Dfile.encoding=UTF-8 option, this way myfaces doesn't convert all non-ascii characters to html entities), myfaces 1.1.3
> Reporter: Tom Pasierb
> Assigned To: Gary VanMatre
> Attachments: some.html, whatever.jsp
>
>
> Clay reads html files assuming ascii encoding. This way it's impossible to have characters other than ascii in templates. They do not display correctly. As indicated on the user mailing list a Reader object should be used for reading templates instead of InputStream. I wrote more about this on shale user mailing list.
> We probably need:
> 1. app wide config option for setting encoding clay should use for reading templates in. Clay would default to this setting unless maybe
> 2. some per file encoding config option was set (something similar to @page pageEncoding directive for jsps)
> I marked it as major as this should be corrected if one wants to develop localized applications with non-ascii characters in html templates.
> As noted by Craig this probably also applies to xml templates, which I haven't tried myself.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/struts/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (SHALE-292) Clay doesn't consider file's encoding
when loading/parsing html templates from hdd
Posted by "Tom Pasierb (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/struts/browse/SHALE-292?page=comments#action_38276 ]
Tom Pasierb commented on SHALE-292:
-----------------------------------
This sounds like a good idea.
However, It would be nice to have an extra application wide config option for loading html templates. The proccessing would look like this:
1) Look at the top of the template for the token comment containing the charset. If not found,
2) look for the app wide config option for template encoding. If found use the encoding for reading the template, If not found use the vm's default "file.encoding".
3) Read the template in with the determined encoding
This way one could have all the templates in a given encoding and unless there was <!-- ### clay:page charset="UTF-8" / --> directive at the top of the file, they would be read with the default encoding set in web.xml and one wouldn't have to define this config in each and every template file. If no <!-- ### clay:page charset="UTF-8" / --> directive was defined in web.xml (null) then clay would fall back to vm'a default file.encoding
How about this?
> Clay doesn't consider file's encoding when loading/parsing html templates from hdd
> ----------------------------------------------------------------------------------
>
> Key: SHALE-292
> URL: http://issues.apache.org/struts/browse/SHALE-292
> Project: Shale
> Issue Type: Bug
> Components: Clay
> Affects Versions: 1.0.0, 1.0.1, 1.0.2, 1.0.4-SNAPSHOT, 1.0.3
> Environment: windows xp, tomcat 5.5 (started with -Dfile.encoding=UTF-8 option, this way myfaces doesn't convert all non-ascii characters to html entities), myfaces 1.1.3
> Reporter: Tom Pasierb
> Assigned To: Gary VanMatre
> Attachments: some.html, whatever.jsp
>
>
> Clay reads html files assuming ascii encoding. This way it's impossible to have characters other than ascii in templates. They do not display correctly. As indicated on the user mailing list a Reader object should be used for reading templates instead of InputStream. I wrote more about this on shale user mailing list.
> We probably need:
> 1. app wide config option for setting encoding clay should use for reading templates in. Clay would default to this setting unless maybe
> 2. some per file encoding config option was set (something similar to @page pageEncoding directive for jsps)
> I marked it as major as this should be corrected if one wants to develop localized applications with non-ascii characters in html templates.
> As noted by Craig this probably also applies to xml templates, which I haven't tried myself.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/struts/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SHALE-292) Clay doesn't consider file's encoding
when loading/parsing html templates from hdd
Posted by "Tom Pasierb (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/struts/browse/SHALE-292?page=all ]
Tom Pasierb updated SHALE-292:
------------------------------
Attachment: whatever.jsp
some.html
Both files are encoded in utf-8 encoding.
> Clay doesn't consider file's encoding when loading/parsing html templates from hdd
> ----------------------------------------------------------------------------------
>
> Key: SHALE-292
> URL: http://issues.apache.org/struts/browse/SHALE-292
> Project: Shale
> Issue Type: Bug
> Components: Clay
> Affects Versions: 1.0.0, 1.0.1, 1.0.2, 1.0.4-SNAPSHOT, 1.0.3
> Environment: windows xp, tomcat 5.5 (started with -Dfile.encoding=UTF-8 option, this way myfaces doesn't convert all non-ascii characters to html entities), myfaces 1.1.3
> Reporter: Tom Pasierb
> Attachments: some.html, whatever.jsp
>
>
> Clay reads html files assuming ascii encoding. This way it's impossible to have characters other than ascii in templates. They do not display correctly. As indicated on the user mailing list a Reader object should be used for reading templates instead of InputStream. I wrote more about this on shale user mailing list.
> We probably need:
> 1. app wide config option for setting encoding clay should use for reading templates in. Clay would default to this setting unless maybe
> 2. some per file encoding config option was set (something similar to @page pageEncoding directive for jsps)
> I marked it as major as this should be corrected if one wants to develop localized applications with non-ascii characters in html templates.
> As noted by Craig this probably also applies to xml templates, which I haven't tried myself.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/struts/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira