You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Stefano Mazzocchi <st...@apache.org> on 2003/03/11 12:48:19 UTC
Encoding problems
The new webapp welcome page contains a copyright character which is not
encoded as the default HTML entity © or the usual &xxx; char, but
it's directly copied in the proper encoding.
The offending char is contained in the welcome.xslt stylesheet that is
encoded as ISO-8859-1.
The pipeline does
- welcome.xml -> ISO-8859-1
- welcome.xslt -> ISO-8859-1
- xhtml serializer -> UTF-8
the results are indeed encoded using UTF-8, thus the copyright sign ends
up being 16 bits (UTF-8 is a clever mixing of 8bit and 16bit char
encoding that was done for easy back compatibility and compression since
most text is on the lower 8bit spectrum nowadays, UTF-16 is more even in
that respect, but nobody uses it because text is normally half as big)
On MacOSX, the results are interesting:
- mozilla 1.3b (20030212) displays the correct encoding
- safari 1.0b(v60) doesn't
- camino 0.7 (2003030613) displays the correct encoding
- IE 5.2.2 (5010.1) doesn't
I traced the problem down to the fact that, apparently, both IE and
Safari are *NOT* able to understand the encoding from the starting XML PI.
On the other hand, by placing
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
the server creates an HTTP header that instructs the user-agent about
the encoding. This solved the encoding problem on *all* browsers.
Results:
1) this is *NOT* a cocoon issue
2) be aware of the fact that some user-agents do not parse the XML PI to
get the encoding, but only the HTTP headers.
NOTES:
1) there is no clear indication on the XHTML specification about how
user-agents have to guess the encoding
2) there is no indication on what Mime-type the XHTML content should have.
These problems reflect the lack of direct collaboration between the IETF
and W3C on XML/HTTP relationship. Unfortunately, this is only going to
get worse. So be prepared, expecially for severely internationalized
content.
Stefano.
Re: Encoding problems
Posted by Santiago Gala <sg...@hisitech.com>.
Gianugo Rabellino wrote:
> Santiago Gala wrote:
>
>>
>> Last-Modified: could be filled with this (the aggregated semantics for
>> it is clear, I think: the highest modification date in the compound
>> validity object). For objects implementing "dynamic" retrieval, like
>> xsp sheets, they should set "now" and a close to zero expiration date
>>
>> This would allow for requests with If-Modified-Since: set to return a
>> small body (304), saving a lot for quasi-static GETable resources.
>
>
> You might want to read the recent threads on proxy friendly cache
> headers. We were talking exactly about this, and I committed a few
> patches on the road of proxy compliance. If you feel like helping,
> you're most welcome. :-)
>
I have been reading the one from february. I'll look for more.
I may well need Cocoon's cache for use in a project soon. If It happens,
I will look at it carefully and report back.
I think the direction you are taking is right. I would map/augment the
semantics of Cocoon's cache to support propagation of proxying related
properties through the pipeline. As I said, what I like from it is that
Cocoon's cache preserves and aggregates very nicely caching information,
so that a given pipeline can have correct behaviour, no matter which
element of it changes.
> Ciao,
>
Ciao, see you soon
Re: Encoding problems
Posted by Gianugo Rabellino <gi...@apache.org>.
Santiago Gala wrote:
>
> Last-Modified: could be filled with this (the aggregated semantics for
> it is clear, I think: the highest modification date in the compound
> validity object). For objects implementing "dynamic" retrieval, like xsp
> sheets, they should set "now" and a close to zero expiration date
>
> This would allow for requests with If-Modified-Since: set to return a
> small body (304), saving a lot for quasi-static GETable resources.
You might want to read the recent threads on proxy friendly cache
headers. We were talking exactly about this, and I committed a few
patches on the road of proxy compliance. If you feel like helping,
you're most welcome. :-)
Ciao,
--
Gianugo Rabellino
Pro-netics s.r.l.
http://www.pro-netics.com
RE: Encoding problems
Posted by Carsten Ziegeler <cz...@s-und-n.de>.
Santiago Gala wrote:
>
>
> Last-Modified: could be filled with this (the aggregated semantics for
> it is clear, I think: the highest modification date in the compound
> validity object). For objects implementing "dynamic" retrieval, like xsp
> sheets, they should set "now" and a close to zero expiration date
>
> This would allow for requests with If-Modified-Since: set to return a
> small body (304), saving a lot for quasi-static GETable resources.
>
> quoting from 2fc2616 (Last-Modified):
>
> > The exact meaning of this header field depends on the
> implementation of the origin
> > server and the nature of the original resource. For files, it may be
> just the file
> > System last-modified time. For entities with dynamically
> included parts, it may be the
> > most recent of the set of last-modify times for its component
> parts. For database gateways,
> > it may be the last-update time stamp of the record. For
> virtual objects, it may be the last
> > time the internal state changed.
>
> Specific Cache validities could allow for extended semantics, like
> expiration dates, proxy directives to avoid caching out of the container
> (thus revalidating requests every time "must-revalidate") or dictating a
> "max-age", etc.
>
Yes, ok - should be possible.
> Note: I have not seen the cache code in at least a year, I'm referring
> to Ricardo's design ideas and availability just before cocoon-2.0
>
Yes, I know them :) because I used them as a starting point for implementing
the cache.
> Also note that I always suggest this in the worst possible moment (at
> beta stabilization time :-(
Oh don't worry about that, we can handle this :)
Carsten
Re: Encoding problems
Posted by Santiago Gala <sg...@hisitech.com>.
Carsten Ziegeler wrote:
> Santiago Gala wrote:
>
>>Re: cache comments later:
>>
>> When I was discussing Cocoon (then new) cache ideas with Ricardo
>>Rocha, (so around winter 2000/spring 2001) I sent an idea to Cocoon-dev
>>about propagating the CacheValidity to set the Expires in the seriazed
>>output. It looked like a clean (the only clean way I have found to date
>>in any famework) to set the "expires:" header in a web publishing
>>framework. AFAIK it got lost in the sea of bits. If any of you set on
>>fight against the cache, it could be very worthwhile. And I would have
>>contributed another bits to Cocoon ;-)
>>
>
> In theory it is possible to set the expires header and to propagate the
> cachevalidity. But (currently) the cachevalidity does not hold any
> expires information. In most cases it contains a last modification
> date of the used files (xml/xslt).
>
> Carsten
Last-Modified: could be filled with this (the aggregated semantics for
it is clear, I think: the highest modification date in the compound
validity object). For objects implementing "dynamic" retrieval, like xsp
sheets, they should set "now" and a close to zero expiration date
This would allow for requests with If-Modified-Since: set to return a
small body (304), saving a lot for quasi-static GETable resources.
quoting from 2fc2616 (Last-Modified):
> The exact meaning of this header field depends on the implementation of the origin
> server and the nature of the original resource. For files, it may be
just the file
> System last-modified time. For entities with dynamically included parts, it may be the
> most recent of the set of last-modify times for its component parts. For database gateways,
> it may be the last-update time stamp of the record. For virtual objects, it may be the last
> time the internal state changed.
Specific Cache validities could allow for extended semantics, like
expiration dates, proxy directives to avoid caching out of the container
(thus revalidating requests every time "must-revalidate") or dictating a
"max-age", etc.
Note: I have not seen the cache code in at least a year, I'm referring
to Ricardo's design ideas and availability just before cocoon-2.0
Also note that I always suggest this in the worst possible moment (at
beta stabilization time :-(
RE: Encoding problems
Posted by Carsten Ziegeler <cz...@s-und-n.de>.
Santiago Gala wrote:
> Re: cache comments later:
>
> When I was discussing Cocoon (then new) cache ideas with Ricardo
> Rocha, (so around winter 2000/spring 2001) I sent an idea to Cocoon-dev
> about propagating the CacheValidity to set the Expires in the seriazed
> output. It looked like a clean (the only clean way I have found to date
> in any famework) to set the "expires:" header in a web publishing
> framework. AFAIK it got lost in the sea of bits. If any of you set on
> fight against the cache, it could be very worthwhile. And I would have
> contributed another bits to Cocoon ;-)
>
In theory it is possible to set the expires header and to propagate the
cachevalidity. But (currently) the cachevalidity does not hold any
expires information. In most cases it contains a last modification
date of the used files (xml/xslt).
Carsten
Re: Encoding problems
Posted by Santiago Gala <sg...@hisitech.com>.
Stefano Mazzocchi wrote:
> Vadim Gritsenko wrote:
>
>> Stefano Mazzocchi wrote:
>> ...
>>
>>> On the other hand, by placing
>>>
>>> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
>>>
>>> the server creates an HTTP header that instructs the user-agent about
>>> the encoding. This solved the encoding problem on *all* browsers.
>>
>>
>>
>>
>> Why not simply patch the serializer to set proper content-type header?
>
>
> D'oh! You're soooooo right. :))
>
> I'll put it on my impossibly long todo list.
>
Re: cache comments later:
When I was discussing Cocoon (then new) cache ideas with Ricardo
Rocha, (so around winter 2000/spring 2001) I sent an idea to Cocoon-dev
about propagating the CacheValidity to set the Expires in the seriazed
output. It looked like a clean (the only clean way I have found to date
in any famework) to set the "expires:" header in a web publishing
framework. AFAIK it got lost in the sea of bits. If any of you set on
fight against the cache, it could be very worthwhile. And I would have
contributed another bits to Cocoon ;-)
Regards,
Santiago
Re: Encoding problems
Posted by Stefano Mazzocchi <st...@apache.org>.
Vadim Gritsenko wrote:
> Stefano Mazzocchi wrote:
> ...
>
>> On the other hand, by placing
>>
>> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
>>
>> the server creates an HTTP header that instructs the user-agent about
>> the encoding. This solved the encoding problem on *all* browsers.
>
>
>
> Why not simply patch the serializer to set proper content-type header?
D'oh! You're soooooo right. :))
I'll put it on my impossibly long todo list.
S.
Re: Encoding problems
Posted by Vadim Gritsenko <va...@verizon.net>.
Stefano Mazzocchi wrote:
...
> On the other hand, by placing
>
> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
>
> the server creates an HTTP header that instructs the user-agent about
> the encoding. This solved the encoding problem on *all* browsers.
Why not simply patch the serializer to set proper content-type header?
Vadim
Re: Encoding problems
Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Tuesday, March 11, 2003, at 11:48 AM, Stefano Mazzocchi wrote:
> The new webapp welcome page contains a copyright character which is
> not encoded as the default HTML entity © or the usual &xxx; char,
> but it's directly copied in the proper encoding.
>
<snip/>
Many thanks for your analysis and confirmation of this issue.
regards Jeremy