You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@sling.apache.org by Chris Millar <ch...@millr.org> on 2017/11/29 23:45:15 UTC
Sling Model Exporter appears to set incorrect response content-type
I've constructed a Sling Model Exporter with very basic options [0], and it
seems to be outputting:
Content-Type: application/json; charset=ISO-8859-1
This breaks the JSON RFC [1] as the default character encoding. It should
be:
Content-Type: application/json; charset=UTF-8
I'm hoping there's a solution to this, or that I'm missing something
obvious. Any help is greatly appreciated.
[0] https://gist.github.com/auniverseaway/c8b7b9013b5ecdd9c790253d194d17b6
[1] https://tools.ietf.org/html/rfc7159#section-8.1
Re: Sling Model Exporter appears to set incorrect response content-type
Posted by Justin Edelson <ju...@justinedelson.com>.
Thanks. I think I see the problem. This is a bug in Jetty.
https://github.com/eclipse/jetty.project/blob/master/jetty-http/src/main/resources/org/eclipse/jetty/http/encoding.properties
only
includes text/json as mapped to UTF-8. It should also set this for
application/json.
We could certainly choose to work around this in Sling, but it should be
done at a lower level than the ExportServlet.
Regards,
Justin
On Wed, Nov 29, 2017 at 11:19 PM Chris Millar <ch...@millr.org> wrote:
> "Hitting the JCR directly" I.E. No Exporter:
> http://localhost:8080/content/bar.0.json for a JCR dump via Sling.
>
> It looks like this is a result of the Export Servlet not setting any
> character encoding, and only setting the content type. Both are needed,
> otherwise `charset` defaults to ISO-8859-1 which comes from
> javax.servlet.http.HttpServletResponse.
>
> I have put together a demo project [0] to see the before results, as well
> as a fork of sling.model.impl to see the after [1] when character encoding
> properly set.
>
> Install the demo, navigate to [2] and note the charset in Content-Type
> header. It's incorrect (ISO-8859-1).
> Once done, install the POC branch of models impl (and snapshot
> dependencies) to see how the response has changed.
>
> This is a bare-minimum solution [3]. Really, the ExportServlet should
> detect the proper character encoding. This could come from either setting
> in the request on the model (passed through the mixin?), or getting it from
> other means.
>
> [0] https://github.com/auniverseaway/sling-model-charset
> [1]
>
> https://github.com/auniverseaway/sling-org-apache-sling-models-impl/tree/POC_Character_Encoding
> [2] http://localhost:8080/content/bar.model.json
> [3]
>
> https://github.com/auniverseaway/sling-org-apache-sling-models-impl/blob/POC_Character_Encoding/src/main/java/org/apache/sling/models/impl/ExportServlet.java#L95
>
> On Wed, Nov 29, 2017 at 7:04 PM, Justin Edelson <ju...@justinedelson.com>
> wrote:
>
> > I'm not sure what you mean by "Hitting the JCR directly..." since JCR has
> > no HTTP support.
> >
> > In any case, as I said, the ExportServlet doesn't do any special handling
> > with the content type and the ITs show that application/json is being
> set,
> > so I would suggest using a debugger against your instance to see where
> the
> > charset is being introduced.
> >
> > Regards,
> > Justin
> >
> > On Wed, Nov 29, 2017 at 7:30 PM Chris Millar <ch...@millr.org> wrote:
> >
> > > I'm accessing Sling (AEM) directly on port 4502. I can put together a
> > test
> > > project with Sling and see if it makes a difference.
> > >
> > > If you look at the getResponseContentType method [0], you can see this
> > does
> > > not include the charset.
> > >
> > > Is it getting set somewhere else or being defaulted? Hitting the JCR
> > > directly gives me the correct charset.
> > >
> > > Is there no way I can get to the response object to set this in the
> > model?
> > > Looking at the Impl, it didn't look like it.
> > >
> > > [0]
> > >
> > > https://sling.apache.org/apidocs/sling9/org/apache/sling/api/
> > SlingHttpServletRequest.html#getResponseContentType
> > >
> > > On Wed, Nov 29, 2017 at 5:14 PM, Justin Edelson <
> > justin@justinedelson.com>
> > > wrote:
> > >
> > > > Hi Chris,
> > > > Are you sure that the charset is being added by Sling, i.e. are you
> > > > accessing Sling directly or via an Apache server?
> > > >
> > > > The Export Servlet doesn't actually try to infer anything about the
> > > content
> > > > type header. It just sets it to request.getResponseContentType(). See
> > > > [1]. If you trace that back far enough, it ultimately comes from the
> > > > MimeTypeService which I don't believe would ever include a charset.
> In
> > > the
> > > > ITs this comes back as just application/json.
> > > >
> > > > Regards,
> > > > Justin
> > > >
> > > > [1]
> > > > https://github.com/apache/sling-org-apache-sling-models-
> > > > impl/blob/master/src/main/java/org/apache/sling/models/
> > > > impl/ExportServlet.java#L110
> > > > [2]
> > > > https://github.com/apache/sling-org-apache-sling-models-
> > > > integration-tests/blob/master/src/test/java/org/apache/
> > > > sling/models/testing/exporter/ExporterIT.java#L287
> > > >
> > > > On Wed, Nov 29, 2017 at 6:45 PM Chris Millar <ch...@millr.org>
> wrote:
> > > >
> > > > > I've constructed a Sling Model Exporter with very basic options
> [0],
> > > and
> > > > it
> > > > > seems to be outputting:
> > > > >
> > > > > Content-Type: application/json; charset=ISO-8859-1
> > > > >
> > > > > This breaks the JSON RFC [1] as the default character encoding. It
> > > should
> > > > > be:
> > > > >
> > > > > Content-Type: application/json; charset=UTF-8
> > > > >
> > > > > I'm hoping there's a solution to this, or that I'm missing
> something
> > > > > obvious. Any help is greatly appreciated.
> > > > >
> > > > > [0]
> > > https://gist.github.com/auniverseaway/c8b7b9013b5ecdd9c790253d194d17
> > > > b6
> > > > > [1] https://tools.ietf.org/html/rfc7159#section-8.1
> > > > >
> > > >
> > >
> >
>
Re: Sling Model Exporter appears to set incorrect response content-type
Posted by Chris Millar <ch...@millr.org>.
"Hitting the JCR directly" I.E. No Exporter:
http://localhost:8080/content/bar.0.json for a JCR dump via Sling.
It looks like this is a result of the Export Servlet not setting any
character encoding, and only setting the content type. Both are needed,
otherwise `charset` defaults to ISO-8859-1 which comes from
javax.servlet.http.HttpServletResponse.
I have put together a demo project [0] to see the before results, as well
as a fork of sling.model.impl to see the after [1] when character encoding
properly set.
Install the demo, navigate to [2] and note the charset in Content-Type
header. It's incorrect (ISO-8859-1).
Once done, install the POC branch of models impl (and snapshot
dependencies) to see how the response has changed.
This is a bare-minimum solution [3]. Really, the ExportServlet should
detect the proper character encoding. This could come from either setting
in the request on the model (passed through the mixin?), or getting it from
other means.
[0] https://github.com/auniverseaway/sling-model-charset
[1]
https://github.com/auniverseaway/sling-org-apache-sling-models-impl/tree/POC_Character_Encoding
[2] http://localhost:8080/content/bar.model.json
[3]
https://github.com/auniverseaway/sling-org-apache-sling-models-impl/blob/POC_Character_Encoding/src/main/java/org/apache/sling/models/impl/ExportServlet.java#L95
On Wed, Nov 29, 2017 at 7:04 PM, Justin Edelson <ju...@justinedelson.com>
wrote:
> I'm not sure what you mean by "Hitting the JCR directly..." since JCR has
> no HTTP support.
>
> In any case, as I said, the ExportServlet doesn't do any special handling
> with the content type and the ITs show that application/json is being set,
> so I would suggest using a debugger against your instance to see where the
> charset is being introduced.
>
> Regards,
> Justin
>
> On Wed, Nov 29, 2017 at 7:30 PM Chris Millar <ch...@millr.org> wrote:
>
> > I'm accessing Sling (AEM) directly on port 4502. I can put together a
> test
> > project with Sling and see if it makes a difference.
> >
> > If you look at the getResponseContentType method [0], you can see this
> does
> > not include the charset.
> >
> > Is it getting set somewhere else or being defaulted? Hitting the JCR
> > directly gives me the correct charset.
> >
> > Is there no way I can get to the response object to set this in the
> model?
> > Looking at the Impl, it didn't look like it.
> >
> > [0]
> >
> > https://sling.apache.org/apidocs/sling9/org/apache/sling/api/
> SlingHttpServletRequest.html#getResponseContentType
> >
> > On Wed, Nov 29, 2017 at 5:14 PM, Justin Edelson <
> justin@justinedelson.com>
> > wrote:
> >
> > > Hi Chris,
> > > Are you sure that the charset is being added by Sling, i.e. are you
> > > accessing Sling directly or via an Apache server?
> > >
> > > The Export Servlet doesn't actually try to infer anything about the
> > content
> > > type header. It just sets it to request.getResponseContentType(). See
> > > [1]. If you trace that back far enough, it ultimately comes from the
> > > MimeTypeService which I don't believe would ever include a charset. In
> > the
> > > ITs this comes back as just application/json.
> > >
> > > Regards,
> > > Justin
> > >
> > > [1]
> > > https://github.com/apache/sling-org-apache-sling-models-
> > > impl/blob/master/src/main/java/org/apache/sling/models/
> > > impl/ExportServlet.java#L110
> > > [2]
> > > https://github.com/apache/sling-org-apache-sling-models-
> > > integration-tests/blob/master/src/test/java/org/apache/
> > > sling/models/testing/exporter/ExporterIT.java#L287
> > >
> > > On Wed, Nov 29, 2017 at 6:45 PM Chris Millar <ch...@millr.org> wrote:
> > >
> > > > I've constructed a Sling Model Exporter with very basic options [0],
> > and
> > > it
> > > > seems to be outputting:
> > > >
> > > > Content-Type: application/json; charset=ISO-8859-1
> > > >
> > > > This breaks the JSON RFC [1] as the default character encoding. It
> > should
> > > > be:
> > > >
> > > > Content-Type: application/json; charset=UTF-8
> > > >
> > > > I'm hoping there's a solution to this, or that I'm missing something
> > > > obvious. Any help is greatly appreciated.
> > > >
> > > > [0]
> > https://gist.github.com/auniverseaway/c8b7b9013b5ecdd9c790253d194d17
> > > b6
> > > > [1] https://tools.ietf.org/html/rfc7159#section-8.1
> > > >
> > >
> >
>
Re: Sling Model Exporter appears to set incorrect response content-type
Posted by Justin Edelson <ju...@justinedelson.com>.
I'm not sure what you mean by "Hitting the JCR directly..." since JCR has
no HTTP support.
In any case, as I said, the ExportServlet doesn't do any special handling
with the content type and the ITs show that application/json is being set,
so I would suggest using a debugger against your instance to see where the
charset is being introduced.
Regards,
Justin
On Wed, Nov 29, 2017 at 7:30 PM Chris Millar <ch...@millr.org> wrote:
> I'm accessing Sling (AEM) directly on port 4502. I can put together a test
> project with Sling and see if it makes a difference.
>
> If you look at the getResponseContentType method [0], you can see this does
> not include the charset.
>
> Is it getting set somewhere else or being defaulted? Hitting the JCR
> directly gives me the correct charset.
>
> Is there no way I can get to the response object to set this in the model?
> Looking at the Impl, it didn't look like it.
>
> [0]
>
> https://sling.apache.org/apidocs/sling9/org/apache/sling/api/SlingHttpServletRequest.html#getResponseContentType
>
> On Wed, Nov 29, 2017 at 5:14 PM, Justin Edelson <ju...@justinedelson.com>
> wrote:
>
> > Hi Chris,
> > Are you sure that the charset is being added by Sling, i.e. are you
> > accessing Sling directly or via an Apache server?
> >
> > The Export Servlet doesn't actually try to infer anything about the
> content
> > type header. It just sets it to request.getResponseContentType(). See
> > [1]. If you trace that back far enough, it ultimately comes from the
> > MimeTypeService which I don't believe would ever include a charset. In
> the
> > ITs this comes back as just application/json.
> >
> > Regards,
> > Justin
> >
> > [1]
> > https://github.com/apache/sling-org-apache-sling-models-
> > impl/blob/master/src/main/java/org/apache/sling/models/
> > impl/ExportServlet.java#L110
> > [2]
> > https://github.com/apache/sling-org-apache-sling-models-
> > integration-tests/blob/master/src/test/java/org/apache/
> > sling/models/testing/exporter/ExporterIT.java#L287
> >
> > On Wed, Nov 29, 2017 at 6:45 PM Chris Millar <ch...@millr.org> wrote:
> >
> > > I've constructed a Sling Model Exporter with very basic options [0],
> and
> > it
> > > seems to be outputting:
> > >
> > > Content-Type: application/json; charset=ISO-8859-1
> > >
> > > This breaks the JSON RFC [1] as the default character encoding. It
> should
> > > be:
> > >
> > > Content-Type: application/json; charset=UTF-8
> > >
> > > I'm hoping there's a solution to this, or that I'm missing something
> > > obvious. Any help is greatly appreciated.
> > >
> > > [0]
> https://gist.github.com/auniverseaway/c8b7b9013b5ecdd9c790253d194d17
> > b6
> > > [1] https://tools.ietf.org/html/rfc7159#section-8.1
> > >
> >
>
Re: Sling Model Exporter appears to set incorrect response content-type
Posted by Chris Millar <ch...@millr.org>.
I'm accessing Sling (AEM) directly on port 4502. I can put together a test
project with Sling and see if it makes a difference.
If you look at the getResponseContentType method [0], you can see this does
not include the charset.
Is it getting set somewhere else or being defaulted? Hitting the JCR
directly gives me the correct charset.
Is there no way I can get to the response object to set this in the model?
Looking at the Impl, it didn't look like it.
[0]
https://sling.apache.org/apidocs/sling9/org/apache/sling/api/SlingHttpServletRequest.html#getResponseContentType
On Wed, Nov 29, 2017 at 5:14 PM, Justin Edelson <ju...@justinedelson.com>
wrote:
> Hi Chris,
> Are you sure that the charset is being added by Sling, i.e. are you
> accessing Sling directly or via an Apache server?
>
> The Export Servlet doesn't actually try to infer anything about the content
> type header. It just sets it to request.getResponseContentType(). See
> [1]. If you trace that back far enough, it ultimately comes from the
> MimeTypeService which I don't believe would ever include a charset. In the
> ITs this comes back as just application/json.
>
> Regards,
> Justin
>
> [1]
> https://github.com/apache/sling-org-apache-sling-models-
> impl/blob/master/src/main/java/org/apache/sling/models/
> impl/ExportServlet.java#L110
> [2]
> https://github.com/apache/sling-org-apache-sling-models-
> integration-tests/blob/master/src/test/java/org/apache/
> sling/models/testing/exporter/ExporterIT.java#L287
>
> On Wed, Nov 29, 2017 at 6:45 PM Chris Millar <ch...@millr.org> wrote:
>
> > I've constructed a Sling Model Exporter with very basic options [0], and
> it
> > seems to be outputting:
> >
> > Content-Type: application/json; charset=ISO-8859-1
> >
> > This breaks the JSON RFC [1] as the default character encoding. It should
> > be:
> >
> > Content-Type: application/json; charset=UTF-8
> >
> > I'm hoping there's a solution to this, or that I'm missing something
> > obvious. Any help is greatly appreciated.
> >
> > [0] https://gist.github.com/auniverseaway/c8b7b9013b5ecdd9c790253d194d17
> b6
> > [1] https://tools.ietf.org/html/rfc7159#section-8.1
> >
>
Re: Sling Model Exporter appears to set incorrect response content-type
Posted by Justin Edelson <ju...@justinedelson.com>.
Hi Chris,
Are you sure that the charset is being added by Sling, i.e. are you
accessing Sling directly or via an Apache server?
The Export Servlet doesn't actually try to infer anything about the content
type header. It just sets it to request.getResponseContentType(). See
[1]. If you trace that back far enough, it ultimately comes from the
MimeTypeService which I don't believe would ever include a charset. In the
ITs this comes back as just application/json.
Regards,
Justin
[1]
https://github.com/apache/sling-org-apache-sling-models-impl/blob/master/src/main/java/org/apache/sling/models/impl/ExportServlet.java#L110
[2]
https://github.com/apache/sling-org-apache-sling-models-integration-tests/blob/master/src/test/java/org/apache/sling/models/testing/exporter/ExporterIT.java#L287
On Wed, Nov 29, 2017 at 6:45 PM Chris Millar <ch...@millr.org> wrote:
> I've constructed a Sling Model Exporter with very basic options [0], and it
> seems to be outputting:
>
> Content-Type: application/json; charset=ISO-8859-1
>
> This breaks the JSON RFC [1] as the default character encoding. It should
> be:
>
> Content-Type: application/json; charset=UTF-8
>
> I'm hoping there's a solution to this, or that I'm missing something
> obvious. Any help is greatly appreciated.
>
> [0] https://gist.github.com/auniverseaway/c8b7b9013b5ecdd9c790253d194d17b6
> [1] https://tools.ietf.org/html/rfc7159#section-8.1
>