You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by "Jay Freeman (saurik)" <sa...@saurik.com> on 2001/03/24 14:05:12 UTC
3/1/2001 FOP0.17 Change for OutputStream vs. Writer Fudged Unicode
I just upgraded my CVS copy of cocoon and one of the websites I host
suddenly started looking stupid :-). My knowledge of character sets is so
limited I am going to avoid trying to describe it (trust me, this is a
*good* thing
especially considering how tired I am).
It isnt an important website, not released to the public yet anyway, so I
left it like this as an example and threw together a simple demonstration
page:
Good: http://www.redoz.com/chars.jsp
Bad: http://www.redoz.com/chars.xsp
It just needs to get run through cocoon for it to go all crazy looking. I
sat around with CVS for a while going back and forth in time until I found
the change that did it in:
CVS: upgrade to support FOP0.17, Cocoon Formatter interface is changed by
this commit
changes.xml:
<action dev="RDG" type="add" due-to="Donald Ball"
due-to-email="balld@webslingerZ.com">
Added "fastcutoff" setting to cocoon.properties to control sendRedirect,
getOutputStream and getWriter behaviour for maximum flexibility and
backward compatibility.
</action>
<action dev="RDG" type="update" due-to="Kelly Campbell"
due-to-email="camk@merlot.channelpoint.com">
Updated Cocoon for compatibility with FOP 0.16 and above. Backward
compatibility to at least FOP 0.13
is still maintained.
WARNING: The Formatter interface has been changed. Instead of taking a
Writer it now takes
an OutputStream. This makes more sense for output types like PDF.
</action>
Of course, it could be that this change FIXED something and that there is
something wrong with those files, in which case does anyone know what that
might be? :-)
Sincerely,
Jay Freeman (saurik)
saurik@saurik.com <ma...@saurik.com>
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: 3/1/2001 FOP0.17 Change for OutputStream vs. Writer Fudged Unicode
Posted by Kelly Campbell <ca...@merlot.channelpoint.com>.
Looks like the encoding for AbstractFormatters comes from a Configurations object's
"encoding" property. I'm not familiar enough with cocoon to tell how to set that
though. Look at the init method on AbstractFormatter.
My patch was a pretty straightforward refactoring. But I now see my mistake.
I wasn't sure how the encoding on the writers got set before. I saw the
getEncoding() method on Formatter, and figured that was what to use.
My mistake was that the original code just used the platform's default encoding:
- StringWriter writer = new StringWriter();
- formatter.format(document, writer, environment);
Just using
page.setContent(new String(outputStream.toByteArray()).getBytes())
should be an equivalent operation to what it was doing before any of my changes.
What I still don't understand is once the byte array is converted to a String in the
default encoding, how does that encoding get propagated to the web client? If the
formatter.getEncoding() is null, then it still sets the Content-Type without a charset
around line 405 of Engine.java.
One way that should fix this with the current code would be to figure out if the
mime-type is text or binary, and if it's text, and the encoding property is null, use the
platform's default encoding, except I can't find a way to get the platform default.
Code:
if (encoding != null) {
page.setContentType(formatter.getMIMEType() + "; charset=" +
encoding);
} else {
// use default character encoding for text mimetypes
if (formatter.getMIMEType() != null &&
formatter.getMIMEType().startsWith("text")) {
// can't find a way to get the platform default encoding
:-(
page.setContentType(formatter.getMIMEType() +
"; charset=UTF-8" );
}
else {
page.setContentType(formatter.getMIMEType());
}
}
Let me know if I'm way off here.
-Kelly
On Sat, Mar 24, 2001 at 06:02:01PM -0600, Jay Freeman saurik" wrote:
> Kelly:
>
> OK, turns out this wouldn't work anyway, as getEncoding() is returning null.
> If you pass a null encoding to String it kind of loses it. Maybe there were
> further changes from your patch that caused this to be a problem. This _is_
> what is causing the problem, however... I finally got it to work with the
> following (ugly) patch:
>
> Index: src/org/apache/cocoon/Engine.java
> ===================================================================
> RCS file: /home/cvspublic/xml-cocoon/src/org/apache/cocoon/Engine.java,v
> retrieving revision 1.54
> diff -u -r1.54 Engine.java
> --- src/org/apache/cocoon/Engine.java 2001/03/01 16:24:20 1.54
> +++ src/org/apache/cocoon/Engine.java 2001/03/24 23:59:45
> @@ -397,7 +397,10 @@
>
> // fill the page bean with content
> if (PROFILE) profiler.startEvent (requestMarker,
> OUTPUTTING);
> - page.setContent(outputStream.toByteArray());
> +
> + encoding = formatter.getEncoding();
> + if (encoding == null) encoding = "UTF-8";
> + page.setContent(new
> String(outputStream.toByteArray(), encoding).getBytes());
>
> // set content type together with encoding if
> appropriate
> encoding = formatter.getEncoding();
>
> Not sure what I could to do make it more efficient....
>
> Sincerely,
> Jay Freeman (saurik)
> saurik@saurik.com
>
> -----Original Message-----
> From: Kelly Campbell [mailto:camk@merlot.channelpoint.com]
> Sent: Saturday, March 24, 2001 2:34 PM
> To: cocoon-dev@xml.apache.org
> Subject: Re: 3/1/2001 FOP0.17 Change for OutputStream vs. Writer Fudged
> Unicode
>
> ...
>
> <<<<<<< Engine.java
> page.setContent(new String(stream.toByteArray(),
> formatter.getEncoding()));
> =======
> page.setContent(outputStream.toByteArray());
> >>>>>>> 1.54
>
> ...
>
> -Kelly
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
--
Kelly A. Campbell Software Engineer
<ca...@merlotxml.org> ChannelPoint, Inc.
<ca...@channelpoint.com> Colorado Springs, Co.
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
RE: 3/1/2001 FOP0.17 Change for OutputStream vs. Writer Fudged Unicode
Posted by "Jay Freeman (saurik)" <sa...@saurik.com>.
Kelly:
OK, turns out this wouldn't work anyway, as getEncoding() is returning null.
If you pass a null encoding to String it kind of loses it. Maybe there were
further changes from your patch that caused this to be a problem. This _is_
what is causing the problem, however... I finally got it to work with the
following (ugly) patch:
Index: src/org/apache/cocoon/Engine.java
===================================================================
RCS file: /home/cvspublic/xml-cocoon/src/org/apache/cocoon/Engine.java,v
retrieving revision 1.54
diff -u -r1.54 Engine.java
--- src/org/apache/cocoon/Engine.java 2001/03/01 16:24:20 1.54
+++ src/org/apache/cocoon/Engine.java 2001/03/24 23:59:45
@@ -397,7 +397,10 @@
// fill the page bean with content
if (PROFILE) profiler.startEvent (requestMarker,
OUTPUTTING);
- page.setContent(outputStream.toByteArray());
+
+ encoding = formatter.getEncoding();
+ if (encoding == null) encoding = "UTF-8";
+ page.setContent(new
String(outputStream.toByteArray(), encoding).getBytes());
// set content type together with encoding if
appropriate
encoding = formatter.getEncoding();
Not sure what I could to do make it more efficient....
Sincerely,
Jay Freeman (saurik)
saurik@saurik.com
-----Original Message-----
From: Kelly Campbell [mailto:camk@merlot.channelpoint.com]
Sent: Saturday, March 24, 2001 2:34 PM
To: cocoon-dev@xml.apache.org
Subject: Re: 3/1/2001 FOP0.17 Change for OutputStream vs. Writer Fudged
Unicode
...
<<<<<<< Engine.java
page.setContent(new String(stream.toByteArray(),
formatter.getEncoding()));
=======
page.setContent(outputStream.toByteArray());
>>>>>>> 1.54
...
-Kelly
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: 3/1/2001 FOP0.17 Change for OutputStream vs. Writer Fudged Unicode
Posted by Kelly Campbell <ca...@merlot.channelpoint.com>.
It looks like my patch didn't get committed as I submitted it. The particular line which I
think is causing this bug is around line 405 of Engine.java. The following is the conflict
message when I synced to cvs just now and which shows the difference between the version I
submitted vs what got committed:
<<<<<<< Engine.java
page.setContent(new String(stream.toByteArray(), formatter.getEncoding()));
=======
page.setContent(outputStream.toByteArray());
>>>>>>> 1.54
The top line is what I submitted, the second is what was committed.
Jay, try changing this line in Engine.java to use the new String with the encoding and see
if it fixes your problem.
-Kelly
On Sat, Mar 24, 2001 at 07:05:12AM -0600, Jay Freeman saurik" wrote:
> I just upgraded my CVS copy of cocoon and one of the websites I host
> suddenly started looking stupid :-). My knowledge of character sets is so
> limited I am going to avoid trying to describe it (trust me, this is a
> *good* thing
especially considering how tired I am).
>
> It isnt an important website, not released to the public yet anyway, so I
> left it like this as an example and threw together a simple demonstration
> page:
>
> Good: http://www.redoz.com/chars.jsp
> Bad: http://www.redoz.com/chars.xsp
>
> It just needs to get run through cocoon for it to go all crazy looking. I
> sat around with CVS for a while going back and forth in time until I found
> the change that did it in:
>
> CVS: upgrade to support FOP0.17, Cocoon Formatter interface is changed by
> this commit
>
> changes.xml:
> <action dev="RDG" type="add" due-to="Donald Ball"
> due-to-email="balld@webslingerZ.com">
> Added "fastcutoff" setting to cocoon.properties to control sendRedirect,
> getOutputStream and getWriter behaviour for maximum flexibility and
> backward compatibility.
> </action>
> <action dev="RDG" type="update" due-to="Kelly Campbell"
> due-to-email="camk@merlot.channelpoint.com">
> Updated Cocoon for compatibility with FOP 0.16 and above. Backward
> compatibility to at least FOP 0.13
> is still maintained.
> WARNING: The Formatter interface has been changed. Instead of taking a
> Writer it now takes
> an OutputStream. This makes more sense for output types like PDF.
> </action>
>
> Of course, it could be that this change FIXED something and that there is
> something wrong with those files, in which case does anyone know what that
> might be? :-)
>
> Sincerely,
> Jay Freeman (saurik)
> saurik@saurik.com <ma...@saurik.com>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
--
Kelly A. Campbell Software Engineer
<ca...@merlotxml.org> ChannelPoint, Inc.
<ca...@channelpoint.com> Colorado Springs, Co.
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org