You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tapestry.apache.org by Alex Kotchnev <ak...@gmail.com> on 2013/05/22 05:21:31 UTC
Tapestry template encoding and GAE
I have an application on Tapestry 5.3.6 that is deployed to GAE 1.8.0. The
problem I have seems to be appengine specific (and not necessarily tapestry
specific) as I have older versions of this application running on GAE and
rendering the locale specific templates normally (whereas now the older
versions have the same problem).
I have some locale-specific templates (e.g. foo.tml, foo_bg.tml and
foo_ru.tml). When I run the application in the local dev app server, the
locale specific templates render normally. However, when I deploy the
application to the appengine, the locale specific templates render the
characters as question marks (???), e.g. compare http://www.zadachite.comvs
http://www.zadachite.com/bg . I wonder if anyone else has run into this
issue running Tapestry on GAE. A few relevant facts / attemps to fix this ,
so far no luck.
I would appreciate any ideas/tips on how others deal w/ this (possibly when
deploying to GAE), as right now my application is sunk.
---------------------------------------------------------------------------------------------------------
1. I explicitly specified the file encoding to UTF-8 in maven :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
<configuration>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
2. I tried setting the file.encoding and DEFAULT_ENCODING properties, as
suggested in
http://stackoverflow.com/questions/11907764/is-there-a-way-to-use-utf-8-with-app-engine
https://developers.google.com/appengine/docs/java/config/appconfig#System_Properties_and_Environment_Variables
3. (This seems to be the root of the problem) In my local environment, the
Charset.defaultCharset() prints "UTF-8"; however, when deployed to GAE, it
prints US-ASCII
4. This seems to be related to
https://issues.apache.org/jira/browse/TAP5-1778
5. When retrieving values from the component messages (locale specific,
e.g. foo_bg.properties), the values seem to be displayed properly (e.g.
notice the menus in the example pages in the app render properly, not as
question marks).
Re: Tapestry template encoding and GAE
Posted by Lance Java <la...@googlemail.com>.
Yes, patching the bug would require a copy of TemplateParserImpl (71 lines)
and SaxTemplateParser (1197 lines)
The great thing about tapestry / tapestry IOC is that you can workaround a
bug like until a fix is provided in a later release.
Much better than rebuilding a custom version of tapestry from tweaked
sources IMHO.
Re: Tapestry template encoding and GAE
Posted by Rural Hunter <ru...@gmail.com>.
I agree. This patch definitely should be in the core. I also have to
apply this patch myself in the similar way.
δΊ 2013/7/20 13:38, Alex Kotchnev ει:
> I thought I'd just drop a note on the list to report on how I dealt with
> this issue.
>
> At first, I started going down the path of building a module that would
> somehow override this service, and in the end it turned out that I had to
> make way too many changes to make it happen.
>
> Today, I decided, screw it, I'd apply the patch from
> https://issues.apache.org/jira/browse/TAP5-1778 , and set up a custom build
> of tapestry that takes care of this. It turns out it's not a big deal at
> all , e.g.
> 1. clone tapestry git repo
> 2. create a branch off the 5.3 branch, apply the patch from the issue
> 3. change the name of the version to 5.3-enc-patch in build.gradle . Run
> ./gradlew build -x test
> 4. declare a dependency on tapestry-core 5.3.7-enc-patch in my project,
> build and deploy to gae
> 5. Ta-da, my newly deployed application on GAE now displays the bulgarian
> and russian templates as it should , no more ???s -
> http://www.zadachite.com/bg
>
> I really think this patch should be a part of tapestry-core : in the normal
> case, it would have no effect. In a case like GAE when I don't have control
> of the underlying platform encoding, this gives a very nice tool to deal
> with that problem.
>
> The only interesting alternative (e.g. one that for example Wicket and
> Gaelyk have taken to deal w/ the GAE perverted default file encoding) is to
> take in the value of the file encoding from a system property (e.g.
> -Dfile.encoding ) instead of from a symbol. In that case, if there are
> other libraries that might not be aware of tapestry-ioc, they could still
> use the system property to decide what encoding to use , and in GAE, one
> can pass in system properties in appengine.xml.
>
> Hopefully a committer can apply this patch to tapestry-core. If not, maybe
> this would be useful to the next person who deploys to GAE and beats his
> head against the wall long enough just to find this posting.
>
> Cheers -
>
>
> On Wed, May 22, 2013 at 7:57 AM, Alex Kotchnev <ak...@gmail.com> wrote:
>
>> Alejandro,
>> unfortunately, adding the XML declaration doesn't help. .
>>
>> This obviously wouldn't be an issue if a) I was running my own JVM,
>> which in most situations where one hosts an application is the case
>> (unfortunately, not the case in PAAS like GAE) or b) Tapestry didn't use
>> the default platform character set.
>>
>> Looking at the patch attached to TAP5-1778, I'm not entirely sure if I
>> can just override a specific service, as the patch appears to touch a whole
>> bunch of classes, which probably means that if I can override this class,
>> I'll probably have a copy a whole bunch of T5 source code to do that. The
>> alternative seems to be to just apply the patch to tapestry-core and deploy
>> my own build.
>>
>> The strange part is that this used to work - when I originally deployed
>> this application to GAE a couple of years ago, the localized templates
>> rendered just fine. I remember reading that GAE did change linux distro
>> underneath their infrastructure (maybe some version of Debian); however,
>> changing the default character set to US-ASCII hardly makes sense for a
>> platform provider. There is an existing GAE issue to which I added some
>> comments - https://code.google.com/p/googleappengine/issues/detail?id=2219 -
>> feel free to add your own :-)
>>
>> Cheers -
>>
>>
>> On Wed, May 22, 2013 at 5:23 AM, Alejandro Scandroli <
>> alejandroscandroli@gmail.com> wrote:
>>
>>> Josh Canfield 've asked a very good question in TAP5-1778.
>>> does Tapestry respect the XML declaration for changing charset?
>>>
>>> Have you tried adding an xml header to your templates?, like:
>>> <?xml version="1.0" encoding="UTF-8" ?>
>>>
>>>
>>> Alejandro.
>>>
>>> On Wed, May 22, 2013 at 9:49 AM, Lance Java <la...@googlemail.com>
>>> wrote:
>>>> I think you've hit the nail on the head and that this relates to
>>> TAP5-1778.
>>>> As a work around, can you put your special chars in a language specific
>>>> properties file? By default, prop files are treated as UTF-8. You can
>>> also
>>>> override the PropertiesFileParser if you need.
>>>>
>>>> Another option is to override the TemplateParser service to use UTF-8.
>>> See
>>>> the patch attached to the issue.
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>>> For additional commands, e-mail: users-help@tapestry.apache.org
>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Tapestry template encoding and GAE
Posted by Alex Kotchnev <ak...@gmail.com>.
I thought I'd just drop a note on the list to report on how I dealt with
this issue.
At first, I started going down the path of building a module that would
somehow override this service, and in the end it turned out that I had to
make way too many changes to make it happen.
Today, I decided, screw it, I'd apply the patch from
https://issues.apache.org/jira/browse/TAP5-1778 , and set up a custom build
of tapestry that takes care of this. It turns out it's not a big deal at
all , e.g.
1. clone tapestry git repo
2. create a branch off the 5.3 branch, apply the patch from the issue
3. change the name of the version to 5.3-enc-patch in build.gradle . Run
./gradlew build -x test
4. declare a dependency on tapestry-core 5.3.7-enc-patch in my project,
build and deploy to gae
5. Ta-da, my newly deployed application on GAE now displays the bulgarian
and russian templates as it should , no more ???s -
http://www.zadachite.com/bg
I really think this patch should be a part of tapestry-core : in the normal
case, it would have no effect. In a case like GAE when I don't have control
of the underlying platform encoding, this gives a very nice tool to deal
with that problem.
The only interesting alternative (e.g. one that for example Wicket and
Gaelyk have taken to deal w/ the GAE perverted default file encoding) is to
take in the value of the file encoding from a system property (e.g.
-Dfile.encoding ) instead of from a symbol. In that case, if there are
other libraries that might not be aware of tapestry-ioc, they could still
use the system property to decide what encoding to use , and in GAE, one
can pass in system properties in appengine.xml.
Hopefully a committer can apply this patch to tapestry-core. If not, maybe
this would be useful to the next person who deploys to GAE and beats his
head against the wall long enough just to find this posting.
Cheers -
On Wed, May 22, 2013 at 7:57 AM, Alex Kotchnev <ak...@gmail.com> wrote:
> Alejandro,
> unfortunately, adding the XML declaration doesn't help. .
>
> This obviously wouldn't be an issue if a) I was running my own JVM,
> which in most situations where one hosts an application is the case
> (unfortunately, not the case in PAAS like GAE) or b) Tapestry didn't use
> the default platform character set.
>
> Looking at the patch attached to TAP5-1778, I'm not entirely sure if I
> can just override a specific service, as the patch appears to touch a whole
> bunch of classes, which probably means that if I can override this class,
> I'll probably have a copy a whole bunch of T5 source code to do that. The
> alternative seems to be to just apply the patch to tapestry-core and deploy
> my own build.
>
> The strange part is that this used to work - when I originally deployed
> this application to GAE a couple of years ago, the localized templates
> rendered just fine. I remember reading that GAE did change linux distro
> underneath their infrastructure (maybe some version of Debian); however,
> changing the default character set to US-ASCII hardly makes sense for a
> platform provider. There is an existing GAE issue to which I added some
> comments - https://code.google.com/p/googleappengine/issues/detail?id=2219 -
> feel free to add your own :-)
>
> Cheers -
>
>
> On Wed, May 22, 2013 at 5:23 AM, Alejandro Scandroli <
> alejandroscandroli@gmail.com> wrote:
>
>> Josh Canfield 've asked a very good question in TAP5-1778.
>> does Tapestry respect the XML declaration for changing charset?
>>
>> Have you tried adding an xml header to your templates?, like:
>> <?xml version="1.0" encoding="UTF-8" ?>
>>
>>
>> Alejandro.
>>
>> On Wed, May 22, 2013 at 9:49 AM, Lance Java <la...@googlemail.com>
>> wrote:
>> > I think you've hit the nail on the head and that this relates to
>> TAP5-1778.
>> >
>> > As a work around, can you put your special chars in a language specific
>> > properties file? By default, prop files are treated as UTF-8. You can
>> also
>> > override the PropertiesFileParser if you need.
>> >
>> > Another option is to override the TemplateParser service to use UTF-8.
>> See
>> > the patch attached to the issue.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
>> For additional commands, e-mail: users-help@tapestry.apache.org
>>
>>
>
Re: Tapestry template encoding and GAE
Posted by Alex Kotchnev <ak...@gmail.com>.
Alejandro,
unfortunately, adding the XML declaration doesn't help. .
This obviously wouldn't be an issue if a) I was running my own JVM,
which in most situations where one hosts an application is the case
(unfortunately, not the case in PAAS like GAE) or b) Tapestry didn't use
the default platform character set.
Looking at the patch attached to TAP5-1778, I'm not entirely sure if I
can just override a specific service, as the patch appears to touch a whole
bunch of classes, which probably means that if I can override this class,
I'll probably have a copy a whole bunch of T5 source code to do that. The
alternative seems to be to just apply the patch to tapestry-core and deploy
my own build.
The strange part is that this used to work - when I originally deployed
this application to GAE a couple of years ago, the localized templates
rendered just fine. I remember reading that GAE did change linux distro
underneath their infrastructure (maybe some version of Debian); however,
changing the default character set to US-ASCII hardly makes sense for a
platform provider. There is an existing GAE issue to which I added some
comments - https://code.google.com/p/googleappengine/issues/detail?id=2219 -
feel free to add your own :-)
Cheers -
On Wed, May 22, 2013 at 5:23 AM, Alejandro Scandroli <
alejandroscandroli@gmail.com> wrote:
> Josh Canfield 've asked a very good question in TAP5-1778.
> does Tapestry respect the XML declaration for changing charset?
>
> Have you tried adding an xml header to your templates?, like:
> <?xml version="1.0" encoding="UTF-8" ?>
>
>
> Alejandro.
>
> On Wed, May 22, 2013 at 9:49 AM, Lance Java <la...@googlemail.com>
> wrote:
> > I think you've hit the nail on the head and that this relates to
> TAP5-1778.
> >
> > As a work around, can you put your special chars in a language specific
> > properties file? By default, prop files are treated as UTF-8. You can
> also
> > override the PropertiesFileParser if you need.
> >
> > Another option is to override the TemplateParser service to use UTF-8.
> See
> > the patch attached to the issue.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
> For additional commands, e-mail: users-help@tapestry.apache.org
>
>
Re: Tapestry template encoding and GAE
Posted by Alejandro Scandroli <al...@gmail.com>.
Josh Canfield 've asked a very good question in TAP5-1778.
does Tapestry respect the XML declaration for changing charset?
Have you tried adding an xml header to your templates?, like:
<?xml version="1.0" encoding="UTF-8" ?>
Alejandro.
On Wed, May 22, 2013 at 9:49 AM, Lance Java <la...@googlemail.com> wrote:
> I think you've hit the nail on the head and that this relates to TAP5-1778.
>
> As a work around, can you put your special chars in a language specific
> properties file? By default, prop files are treated as UTF-8. You can also
> override the PropertiesFileParser if you need.
>
> Another option is to override the TemplateParser service to use UTF-8. See
> the patch attached to the issue.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Tapestry template encoding and GAE
Posted by Andreas Fink <fi...@googlemail.com>.
wow, i didn't know that!
i had problems with encoding of properties in recent t5 projects but maybe maven got in the way or this service did not get used.
cheers,
andi.
On May 23, 2013, at 10:26 , Lance Java wrote:
> You underestimate tapestry :)
>
> Tapestry's PropertiesFileParserImpl assumes UTF-8 properties files and
> performs a native2ascii transformation on the fly
>
> http://tapestry.apache.org/current/apidocs/src-html/org/apache/tapestry5/internal/services/messages/PropertiesFileParserImpl.html#line.79
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Tapestry template encoding and GAE
Posted by Lance Java <la...@googlemail.com>.
You underestimate tapestry :)
Tapestry's PropertiesFileParserImpl assumes UTF-8 properties files and
performs a native2ascii transformation on the fly
http://tapestry.apache.org/current/apidocs/src-html/org/apache/tapestry5/internal/services/messages/PropertiesFileParserImpl.html#line.79
Re: Tapestry template encoding and GAE
Posted by Andreas Fink <fi...@googlemail.com>.
Hi Lance.
Unfortunately this is not the case. Property files are treated ISO 8859-1 while reading/writing.
If you want to have special chars, you have to run the file through native2ascii first.
http://docs.oracle.com/javase/6/docs/api/java/util/Properties.html
On May 22, 2013, at 9:49 , Lance Java wrote:
> I think you've hit the nail on the head and that this relates to TAP5-1778.
>
> As a work around, can you put your special chars in a language specific
> properties file? By default, prop files are treated as UTF-8. You can also
> override the PropertiesFileParser if you need.
>
> Another option is to override the TemplateParser service to use UTF-8. See
> the patch attached to the issue.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tapestry.apache.org
For additional commands, e-mail: users-help@tapestry.apache.org
Re: Tapestry template encoding and GAE
Posted by Lance Java <la...@googlemail.com>.
I think you've hit the nail on the head and that this relates to TAP5-1778.
As a work around, can you put your special chars in a language specific
properties file? By default, prop files are treated as UTF-8. You can also
override the PropertiesFileParser if you need.
Another option is to override the TemplateParser service to use UTF-8. See
the patch attached to the issue.