You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Stefano Mazzocchi <st...@apache.org> on 2003/03/11 12:48:19 UTC

Encoding problems

The new webapp welcome page contains a copyright character which is not 
encoded as the default HTML entity &copy; or the usual &xxx; char, but 
it's directly copied in the proper encoding.

The offending char is contained in the welcome.xslt stylesheet that is 
encoded as ISO-8859-1.

The pipeline does

  - welcome.xml -> ISO-8859-1
  - welcome.xslt -> ISO-8859-1
  - xhtml serializer -> UTF-8

the results are indeed encoded using UTF-8, thus the copyright sign ends 
up being 16 bits (UTF-8 is a clever mixing of 8bit and 16bit char 
encoding that was done for easy back compatibility and compression since 
most text is on the lower 8bit spectrum nowadays, UTF-16 is more even in 
that respect, but nobody uses it because text is normally half as big)

On MacOSX, the results are interesting:

  - mozilla 1.3b (20030212) displays the correct encoding
  - safari 1.0b(v60) doesn't
  - camino 0.7 (2003030613) displays the correct encoding
  - IE 5.2.2 (5010.1) doesn't

I traced the problem down to the fact that, apparently, both IE and 
Safari are *NOT* able to understand the encoding from the starting XML PI.

On the other hand, by placing

  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>

the server creates an HTTP header that instructs the user-agent about 
the encoding. This solved the encoding problem on *all* browsers.

Results:

1) this is *NOT* a cocoon issue
2) be aware of the fact that some user-agents do not parse the XML PI to 
get the encoding, but only the HTTP headers.

NOTES:
1) there is no clear indication on the XHTML specification about how 
user-agents have to guess the encoding
2) there is no indication on what Mime-type the XHTML content should have.

These problems reflect the lack of direct collaboration between the IETF 
and W3C on XML/HTTP relationship. Unfortunately, this is only going to 
get worse. So be prepared, expecially for severely internationalized 
content.

Stefano.


Re: Encoding problems

Posted by Santiago Gala <sg...@hisitech.com>.
Gianugo Rabellino wrote:
> Santiago Gala wrote:
> 
>>
>> Last-Modified: could be filled with this (the aggregated semantics for 
>> it is clear, I think: the highest modification date in the compound 
>> validity object). For objects implementing "dynamic" retrieval, like 
>> xsp sheets, they should set "now" and a close to zero expiration date
>>
>> This would allow for requests with If-Modified-Since: set to return a 
>> small body (304), saving a lot for quasi-static GETable resources.
> 
> 
> You might want to read the recent threads on proxy friendly cache 
> headers. We were talking exactly about this, and I committed a few 
> patches on the road of proxy compliance. If you feel like helping, 
> you're most welcome. :-)
> 

I have been reading the one from february. I'll look for more.

I may well need Cocoon's cache for use in a project soon. If It happens, 
I will look at it carefully and report back.

I think the direction you are taking is right. I would map/augment the 
semantics of Cocoon's cache to support propagation of proxying related 
properties through the pipeline. As I said, what I like from it is that 
Cocoon's cache preserves and aggregates very nicely caching information, 
so that a given pipeline can have correct behaviour, no matter which 
element of it changes.

> Ciao,
> 

Ciao, see you soon


Re: Encoding problems

Posted by Gianugo Rabellino <gi...@apache.org>.
Santiago Gala wrote:
> 
> Last-Modified: could be filled with this (the aggregated semantics for 
> it is clear, I think: the highest modification date in the compound 
> validity object). For objects implementing "dynamic" retrieval, like xsp 
> sheets, they should set "now" and a close to zero expiration date
> 
> This would allow for requests with If-Modified-Since: set to return a 
> small body (304), saving a lot for quasi-static GETable resources.

You might want to read the recent threads on proxy friendly cache 
headers. We were talking exactly about this, and I committed a few 
patches on the road of proxy compliance. If you feel like helping, 
you're most welcome. :-)

Ciao,

-- 
Gianugo Rabellino
Pro-netics s.r.l.
http://www.pro-netics.com


RE: Encoding problems

Posted by Carsten Ziegeler <cz...@s-und-n.de>.
Santiago Gala wrote:
>
>
> Last-Modified: could be filled with this (the aggregated semantics for
> it is clear, I think: the highest modification date in the compound
> validity object). For objects implementing "dynamic" retrieval, like xsp
> sheets, they should set "now" and a close to zero expiration date
>
> This would allow for requests with If-Modified-Since: set to return a
> small body (304), saving a lot for quasi-static GETable resources.
>
> quoting from 2fc2616 (Last-Modified):
>
> >  The exact meaning of this header field depends on the
> implementation of the origin
>  >  server and the nature of the original resource. For files, it may be
> just the file
> >  System last-modified time. For entities with dynamically
> included parts, it may be the
> >  most recent of the set of last-modify times for its component
> parts. For database gateways,
> >  it may be the last-update time stamp of the record. For
> virtual objects, it may be the last
> >  time the internal state changed.
>
> Specific Cache validities could allow for extended semantics, like
> expiration dates, proxy directives to avoid caching out of the container
> (thus revalidating requests every time "must-revalidate") or dictating a
> "max-age", etc.
>
Yes, ok - should be possible.

> Note: I have not seen the cache code in at least a year, I'm referring
> to Ricardo's design ideas and availability just before cocoon-2.0
>
Yes, I know them :) because I used them as a starting point for implementing
the cache.

> Also note that I always suggest this in the worst possible moment (at
> beta stabilization time :-(
Oh don't worry about that, we can handle this :)

Carsten


Re: Encoding problems

Posted by Santiago Gala <sg...@hisitech.com>.
Carsten Ziegeler wrote:
> Santiago Gala wrote:
> 
>>Re: cache comments later:
>>
>>  When I was discussing Cocoon (then new) cache ideas with Ricardo 
>>Rocha, (so around winter 2000/spring 2001) I sent an idea to Cocoon-dev 
>>about propagating the CacheValidity to set the Expires in the seriazed 
>>output. It looked like a clean (the only clean way I have found to date 
>>in any famework) to set the "expires:" header in a web publishing 
>>framework. AFAIK it got lost in the sea of bits. If any of you set on 
>>fight against the cache, it could be very worthwhile. And I would have 
>>contributed another bits to Cocoon ;-)
>>
> 
> In theory it is possible to set the expires header and to propagate the
> cachevalidity. But (currently) the cachevalidity does not hold any
> expires information. In most cases it contains a last modification
> date of the used files (xml/xslt).
> 
> Carsten

Last-Modified: could be filled with this (the aggregated semantics for 
it is clear, I think: the highest modification date in the compound 
validity object). For objects implementing "dynamic" retrieval, like xsp 
sheets, they should set "now" and a close to zero expiration date

This would allow for requests with If-Modified-Since: set to return a 
small body (304), saving a lot for quasi-static GETable resources.

quoting from 2fc2616 (Last-Modified):

>  The exact meaning of this header field depends on the implementation of the origin 
 >  server and the nature of the original resource. For files, it may be 
just the file
>  System last-modified time. For entities with dynamically included parts, it may be the 
>  most recent of the set of last-modify times for its component parts. For database gateways, 
>  it may be the last-update time stamp of the record. For virtual objects, it may be the last 
>  time the internal state changed.

Specific Cache validities could allow for extended semantics, like 
expiration dates, proxy directives to avoid caching out of the container 
(thus revalidating requests every time "must-revalidate") or dictating a 
"max-age", etc.

Note: I have not seen the cache code in at least a year, I'm referring 
to Ricardo's design ideas and availability just before cocoon-2.0

Also note that I always suggest this in the worst possible moment (at 
beta stabilization time :-(


RE: Encoding problems

Posted by Carsten Ziegeler <cz...@s-und-n.de>.
Santiago Gala wrote:
> Re: cache comments later:
> 
>   When I was discussing Cocoon (then new) cache ideas with Ricardo 
> Rocha, (so around winter 2000/spring 2001) I sent an idea to Cocoon-dev 
> about propagating the CacheValidity to set the Expires in the seriazed 
> output. It looked like a clean (the only clean way I have found to date 
> in any famework) to set the "expires:" header in a web publishing 
> framework. AFAIK it got lost in the sea of bits. If any of you set on 
> fight against the cache, it could be very worthwhile. And I would have 
> contributed another bits to Cocoon ;-)
> 
In theory it is possible to set the expires header and to propagate the
cachevalidity. But (currently) the cachevalidity does not hold any
expires information. In most cases it contains a last modification
date of the used files (xml/xslt).

Carsten

Re: Encoding problems

Posted by Santiago Gala <sg...@hisitech.com>.
Stefano Mazzocchi wrote:
> Vadim Gritsenko wrote:
> 
>> Stefano Mazzocchi wrote:
>> ...
>>
>>> On the other hand, by placing
>>>
>>>  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
>>>
>>> the server creates an HTTP header that instructs the user-agent about 
>>> the encoding. This solved the encoding problem on *all* browsers.
>>
>>
>>
>>
>> Why not simply patch the serializer to set proper content-type header?
> 
> 
> D'oh! You're soooooo right. :))
> 
> I'll put it on my impossibly long todo list.
> 

Re: cache comments later:

  When I was discussing Cocoon (then new) cache ideas with Ricardo 
Rocha, (so around winter 2000/spring 2001) I sent an idea to Cocoon-dev 
about propagating the CacheValidity to set the Expires in the seriazed 
output. It looked like a clean (the only clean way I have found to date 
in any famework) to set the "expires:" header in a web publishing 
framework. AFAIK it got lost in the sea of bits. If any of you set on 
fight against the cache, it could be very worthwhile. And I would have 
contributed another bits to Cocoon ;-)

Regards,
     Santiago


Re: Encoding problems

Posted by Stefano Mazzocchi <st...@apache.org>.
Vadim Gritsenko wrote:
> Stefano Mazzocchi wrote:
> ...
> 
>> On the other hand, by placing
>>
>>  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
>>
>> the server creates an HTTP header that instructs the user-agent about 
>> the encoding. This solved the encoding problem on *all* browsers.
> 
> 
> 
> Why not simply patch the serializer to set proper content-type header?

D'oh! You're soooooo right. :))

I'll put it on my impossibly long todo list.

S.


Re: Encoding problems

Posted by Vadim Gritsenko <va...@verizon.net>.
Stefano Mazzocchi wrote:
...

> On the other hand, by placing
>
>  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
>
> the server creates an HTTP header that instructs the user-agent about 
> the encoding. This solved the encoding problem on *all* browsers.


Why not simply patch the serializer to set proper content-type header?

Vadim




Re: Encoding problems

Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Tuesday, March 11, 2003, at 11:48 AM, Stefano Mazzocchi wrote:

> The new webapp welcome page contains a copyright character which is 
> not encoded as the default HTML entity &copy; or the usual &xxx; char, 
> but it's directly copied in the proper encoding.
>

<snip/>

Many thanks for your analysis and confirmation of this issue.


regards Jeremy