You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by co...@eng.sun.com on 1999/11/19 21:58:35 UTC
Re: [PATCH] Tomcat on EBCDIC machines...
> This patch:
>
> Return the HTTP headers in the response as ASCII when running on a non-ASCII machine (i.e. an IBM mainframe).
> - sout.write( headersSB.toString().getBytes() );
> + sout.write( headersSB.toString().getBytes(encoding) );
> }
I would like to understand a bit more what happens.
My understanding is that the problem is that right now Tomcat uses the "default"
encoding, which is the "native" encoding for the platform tomcat is running on.
A simpler patch ( that will also fix the body - probably :-) is to just set the default
encoding ( eventually on the PrintWriter - where it should be set).
The encoding is set in getWriter() - and I think there is a bug in recycle() and new(),
where we get a system property "file.encoding" instead of using the default UTF/ISOxxx.
Can you check if the problem is resolved by changing Response:
characterEncoding = System.getProperty("file.encoding",
Constants.CharacterEncoding.Default);
to
characterEncoding = Constants.CharacterEncoding.Default;
( 3 times)
I think your problem exposed 3 bugs:
- we shouldn't default to the system encoding, but to ISOxxx ( as
specified in HTTP specs - XXX read it again :-)
- right now we return the headers using the same encoding as the body -
instead of using whatever HTTP requires ( we can/should have different encoding
for head/body !).
- a big one - we don't do any content negotiation ( the request can force
a different encoding, and I don't think it's only the servlet that should deal
with that, probably Tomcat needs to set the default out encoding to whatever
the request asks)
Probably only the first one is urgent/important, and if the Response change
doesn't solve the problem I'll just commit your patch and wait for better
ideea.
Costin
Re: [PATCH] Tomcat on EBCDIC machines...
Posted by David Brownell <da...@pacbell.net>.
costin@eng.sun.com wrote:
>
> I think your problem exposed 3 bugs:
>
> - we shouldn't default to the system encoding, but to ISOxxx ( as
> specified in HTTP specs - XXX read it again :-)
That's correct: ISO-8859-1, which is the low eight bits of Unicode.
Applies only to 'text/html' response bodies though (whoever wrote
up that bit of the HTTP spec broke a big chunk of the MIME world).
For MIME in general, all "text/*" content types default to ASCII.
> - right now we return the headers using the same encoding as the body -
> instead of using whatever HTTP requires ( we can/should have different encoding
> for head/body !).
Best to treat the HTTP header as straight ASCII -- it's guaranteed
never to have characters other than that.
> - a big one - we don't do any content negotiation ( the request can force
> a different encoding, and I don't think it's only the servlet that should deal
> with that, probably Tomcat needs to set the default out encoding to whatever
> the request asks)
Application needs to have some say-so here. Request specifies a list
of acceptible encodings for the response, and application chooses.
For example, if ASCII and EUC-JP are accepted, don't use ASCII if the
response is written with Japanese characters.
The servlet API had support for application interaction there, last
time I looked. As of API 1.1 for sure -- and I don't think I saw
any of the relevant methods get deprecated.
> Probably only the first one is urgent/important, and if the Response change
> doesn't solve the problem I'll just commit your patch and wait for better
> ideea.
Second one is a really severe protocol bug; don't ship a Tomcat
milestone without fixing it (and the first one).
- dave
RE: [PATCH] Tomcat on EBCDIC machines...
Posted by "Preston L. Bannister" <pr...@home.com>.
There is something going wrong in the initial request handling that I suspect is due to a problem (somewhere!) in reading the
configuration files.
I've added some println()'s to get closer to the source of the problem.
Hopefully with reading the source the meaning is clear... :).
First the successful startup and first request (on Win32):
---
JAVA_HOME=d:\jdk1.1.8
CLASSPATH=.\classes;.\webserver.jar;.\lib\servlet.jar;.\lib\jasper.jar;.\webpages\WEB-INF\classes\jsp\beans;.\lib\xml.jar;d:\jdk1.1.
8\lib\classes.zip;d:\jdk1.1.8\lib\tools.jar
Tomcat Web Server Version 2.2
Loaded configuration from: file:/D:\Jakarta\build\tomcat\server.xml
Configuring web service using "default"
Container.addServlet name = "default" wrapper = org.apache.tomcat.core.ServletWrapper@24c8ba
Container.addServlet name = "invoker" wrapper = org.apache.tomcat.core.ServletWrapper@24c823
Container.addServlet name = "jsp" wrapper = org.apache.tomcat.core.ServletWrapper@24c78f
Configuring web service using "file:/D:\Jakarta\build\tomcat\webpages/WEB-INF/web.xml"
Configuring web service using "default"
Container.addServlet name = "default" wrapper = org.apache.tomcat.core.ServletWrapper@269fc8
Container.addServlet name = "invoker" wrapper = org.apache.tomcat.core.ServletWrapper@26a5ee
Container.addServlet name = "jsp" wrapper = org.apache.tomcat.core.ServletWrapper@26a55a
Configuring web service using "file:/D:\Jakarta\build\tomcat\examples/WEB-INF/web.xml"
Container.addServlet name = "snoop" wrapper = org.apache.tomcat.core.ServletWrapper@23c85c
Container.addServlet name = "servletToJsp" wrapper = org.apache.tomcat.core.ServletWrapper@23c7a4
Starting tcp endpoint on 8080 with org.apache.tomcat.service.http.HttpConnectionHandler
Starting tcp endpoint on 8007 with org.apache.tomcat.service.Ajp11ConnectionHandler
Starting tcp endpoint on 8009 with org.apache.tomcat.service.connector.Ajp12ConnectionHandler
Starting tcp endpoint on 8008 with org.apache.tomcat.service.connector.Ajp23ConnectionHandler
HttpRequestAdapter.processRequestLine line = GET / HTTP/1.1
RequestMapper.lookupServlet path = "/" wrapper = null
Container.lookupServlet path = "/" result = null
Container.lookupServlet defaultServlet = null
Container.lookupServlet servlets.get name = default
new LookupResult = LookupResult( SP= PI=/ MP=null RS=null SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
Container.lookupServlet new LookupResult = LookupResult( SP= PI=/ MP=null RS=null SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
Context.handleRequest result = LookupResult( SP= PI=/ MP=null RS=null SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
Context.handleRequest wrapper = org.apache.tomcat.core.ServletWrapper@24c8ba
RequestMapper.lookupServlet path = "/" wrapper = null
Container.lookupServlet path = "/" result = null
Container.lookupServlet defaultServlet = null
Container.lookupServlet servlets.get name = default
new LookupResult = LookupResult( SP= PI=/ MP=null RS=null SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
Container.lookupServlet new LookupResult = LookupResult( SP= PI=/ MP=null RS=null SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
RequestMapper.lookupServlet path = "/index.html" wrapper = null
Container.lookupServlet path = "/index.html" result = null
Container.lookupServlet defaultServlet = null
Container.lookupServlet servlets.get name = default
new LookupResult = LookupResult( SP= PI=/index.html MP=null RS=null SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
Container.lookupServlet new LookupResult = LookupResult( SP= PI=/index.html MP=null RS=null
SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
RequestMapper.lookupServlet path = "/index.html" wrapper = null
Container.lookupServlet path = "/index.html" result = null
Container.lookupServlet defaultServlet = null
Container.lookupServlet servlets.get name = default
new LookupResult = LookupResult( SP= PI=/index.html MP=null RS=null SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
Container.lookupServlet new LookupResult = LookupResult( SP= PI=/index.html MP=null RS=null
SW=org.apache.tomcat.core.ServletWrapper@24c8ba)
---
Second the startup (with complaints) and first (failed) request on OS/390:
---
$ sh startup.sh
JAVA_HOME=/usr/lpp/java/J1.1
CLASSPATH=./classes:./webserver.jar:./lib/servlet.jar:./lib/jasper.jar:./webpages/WEB-INF/classes/jsp/beans:./lib/xml.jar:/usr/lpp/j
ava/J1.1/lr
$ Tomcat Web Server Version 2.2
Loaded configuration from: file:/nsa/users/bannisp/nsa/Jakarta/build/tomcat/server.xml
Configuring web service using "default"
config parse: can't read config: SAXParseException: null : 1
msg : Character conversion error: "Unconvertible UTF-8 character beginning with 0xa7" (line number may be too low).
Configuring web service using "file:/nsa/users/bannisp/nsa/Jakarta/build/tomcat/webpages/WEB-INF/web.xml"
config parse: can't read config: SAXParseException: null : 1
msg : Character conversion error: "Unconvertible UTF-8 character beginning with 0xa7" (line number may be too low).
Configuring web service using "default"
config parse: can't read config: SAXParseException: null : 1
msg : Character conversion error: "Unconvertible UTF-8 character beginning with 0xa7" (line number may be too low).
Configuring web service using "file:/nsa/users/bannisp/nsa/Jakarta/build/tomcat/examples/WEB-INF/web.xml"
config parse: can't read config: SAXParseException: null : 1
msg : Character conversion error: "Unconvertible UTF-8 character beginning with 0xa7" (line number may be too low).
Starting tcp endpoint on 8080 with org.apache.tomcat.service.http.HttpConnectionHandler
Starting tcp endpoint on 8007 with org.apache.tomcat.service.Ajp11ConnectionHandler
Starting tcp endpoint on 8009 with org.apache.tomcat.service.connector.Ajp12ConnectionHandler
Starting tcp endpoint on 8008 with org.apache.tomcat.service.connector.Ajp23ConnectionHandler
HttpRequestAdapter.processRequestLine line = GET / HTTP/1.1
RequestMapper.lookupServlet path = "/" wrapper = null
Container.lookupServlet path = "/" result = null
Container.lookupServlet defaultServlet = null
Container.lookupServlet servlets.get name = default
new LookupResult = LookupResult( SP= PI=/ MP=null RS=null SW=null)
Container.lookupServlet new LookupResult = LookupResult( SP= PI=/ MP=null RS=null SW=null)
Context.handleRequest result = LookupResult( SP= PI=/ MP=null RS=null SW=null)
Context.handleRequest wrapper = null
HANDLER THREAD PROBLEM: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.tomcat.core.Context.handleRequest(Context.java:654)
at org.apache.tomcat.core.ContextManager.service(ContextManager.java:443)
at org.apache.tomcat.service.http.HttpConnectionHandler.processConnection(HttpConnectionHandler.java:138)
at org.apache.tomcat.service.TcpConnectionThread.run(TcpEndpoint.java:301)
at java.lang.Thread.run(Thread.java:490)
---
The end problem seems to be that the "default" servlet isn't getting a wrapper. The reason for this is likely that there is some
trouble in reading the configuration files.
It would be nice if I knew where the "config parse:" errors occur...
RE: [PATCH] Tomcat on EBCDIC machines...
Posted by "Preston L. Bannister" <pr...@home.com>.
From: costin@mta2.snfc21.pbi.net
> My understanding is that the problem is that right now Tomcat uses the "default"
> encoding, which is the "native" encoding for the platform tomcat is running on.
Yes, in many places... implicitly.
> A simpler patch ( that will also fix the body - probably :-) is
> to just set the default encoding (eventually on the PrintWriter
> - where it should be set).
Likely that would solve many things, bit IMHO is the wrong solution.
As Tomcat is meant to be embedded, this could break the container.
> The encoding is set in getWriter() - and I think there is a bug
> in recycle() and new(), where we get a system property
> "file.encoding" instead of using the default UTF/ISOxxx.
>
> Can you check if the problem is resolved by changing Response:
>
> characterEncoding = System.getProperty("file.encoding",
> Constants.CharacterEncoding.Default);
> to
> characterEncoding = Constants.CharacterEncoding.Default;
> ( 3 times)
Did it - but cannot yet tell if it helped on OS/390.
At least it didn't break anything on Win32 :).
There is some problem on startup and the "default" servlet's wrapper does not get set. I will describe this in a seperate email.
> I think your problem exposed 3 bugs:
>
> - we shouldn't default to the system encoding, but to ISOxxx ( as
> specified in HTTP specs - XXX read it again :-)
I read it. I just *wish* that it said UTF-8 :).
> - right now we return the headers using the same encoding as the body -
> instead of using whatever HTTP requires ( we can/should have different encoding
> for head/body !).
Right now the headers are returned in the default (implicit) encoding used by the constructor String(byte[]), which is what the
patch changes.