You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Bob Paulin <bo...@bobpaulin.com> on 2018/08/16 13:29:21 UTC
Re: Content-Length Header being used previous HTTP Response
Agree that removeHeader inbetween the routes fixes the problem. It's
great experience having to go through all the code to figure out a
content-length header mismatch is the issue since the request just
hangs. I ran into it again with the following scenerio using the
automatic json binding in the REST DSL with stream caching enabled.
this.getContext().setStreamCaching(true);
restConfiguration().component("netty4-http").port(8282).bindingMode(RestBindingMode.json);
rest("/bind").post()
.type(InputDTO.class)
.outType(OutputDTO.class)
.to("direct:bind");
from("direct:bind")
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
.setHeader("Accept", constant("application/json"))
.setHeader(Exchange.HTTP_PATH, constant(""))
.setHeader(Exchange.CONTENT_TYPE,
constant("application/json"))
.process(new Processor() {
public void process(Exchange exchange) throws
Exception {
InputDTO body =
exchange.getIn().getBody(InputDTO.class);
body.setDescription("SomethingElse");
}
})
.marshal().json(JsonLibrary.Jackson)
.to("http4://localhost/realservice?bridgeEndpoint=true")
.unmarshal().json(JsonLibrary.Jackson,
FinalOutputDTO.class);
The json coming into /bind is pretty printed so includes spaces and
returns. After it's marshaled to an object and the object is modified
the Content-length will change. Seems like removing the content-length
header in the DataFormat used to do the marshal ling or perhaps
providing an option in the HTTP Client to ignore the content-header
would be useful. Again willing to help collaborate on a patch but I'd
like some input on where it would fit best. Currently thinking either
in the DataFormat marshal methods and/or the HttpProducer.
- Bob
On 2017/09/20 13:13:06, Claus Ibsen <c....@gmail.com> wrote:
> You can remove the header before calling the http endpoints, with>
> removeHeader(Exchange.CONTENT_LENGTH)>
>
> On Sat, Sep 16, 2017 at 9:03 PM, Bob Paulin <bo...@bobpaulin.com> wrote:>
> > Hi,>
> >>
> > I have a camel rest route like the following>
> >>
> > rest("/api/test")>
> > .post()>
> > .to("direct:test");>
> >>
> > from("direct:test").>
> >>
> >
.toD("http4://localhost/test?bridgeEndpoint=true&throwExceptionOnFailure=false")>
> >>
> >
.toD("http4://localhost/test2?bridgeEndpoint=true&throwExceptionOnFailure=false");>
> >>
> >>
> > Trouble occurs when the http4://localhost/test route sets a>
> > content-length header it gets used for http4://localhost/test2. There>
> > are some web servers that use that header to determine request size>
> > which if it never gets to the needed length will never close and
after a>
> > while timeout. This happens despite content-length being in the>
> > HttpHeaderFilterStrategy list because in the HttpProducer class
there is>
> > code like this. So an input stream will still use the header.>
> >>
> > // fallback as input stream>
> > if (answer == null) {>
> > // force the body as an input stream since this>
> > is the fallback>
> > InputStream is =>
> > in.getMandatoryBody(InputStream.class);>
> > String length =>
> > in.getHeader(Exchange.CONTENT_LENGTH, String.class);>
> > InputStreamEntity entity = null;>
> > if (ObjectHelper.isEmpty(length) ) {>
> > entity = new InputStreamEntity(is, -1);>
> > } else {>
> > entity = new InputStreamEntity(is,>
> > Long.parseLong(length));>
> > }>
> > if (contentType != null) {>
> > entity.setContentType(contentType.toString());>
> > }>
> > answer = entity;>
> > }>
> >>
> > I'm assuming there are cases where I might want to explicitly provide>
> > the content-length to the request entity. But I'd like to be able to>
> > turn it off so it always falls back to -1 (chunked).>
> >>
> > I'm working around the issue by doing this:>
> >>
> > from("direct:test").>
> >>
> >
.toD("http4://localhost/test?bridgeEndpoint=true&throwExceptionOnFailure=false")>
> >>
> > .setHeader("Content-Length", constant(""))>
> >>
> >
.toD("http4://localhost/test2?bridgeEndpoint=true&throwExceptionOnFailure=false");>
> >>
> > But I'd like to suggest creating a more global way to ignore the>
> > Content-Length header:>
> >>
> > // fallback as input stream>
> > if (answer == null) {>
> > // force the body as an input stream since this>
> > is the fallback>
> > InputStream is =>
> > in.getMandatoryBody(InputStream.class);>
> > String length =>
> > in.getHeader(Exchange.CONTENT_LENGTH, String.class);>
> > InputStreamEntity entity = null;>
> > if (ObjectHelper.isEmpty(length) ||>
> > getEndpoint().getComponent().isForceChunkedRequestEntity()) {>
> > entity = new InputStreamEntity(is, -1);>
> > } else {>
> > entity = new InputStreamEntity(is,>
> > Long.parseLong(length));>
> > }>
> > if (contentType != null) {>
> > entity.setContentType(contentType.toString());>
> > }>
> > answer = entity;>
> > }>
> >>
> >>
> > Thoughts on this approach? How are other folks dealing with this?>
> > Happy to work on a contribution/patch but I wanted to discuss before>
> > creating a jira.>
> >>
> >>
> > - Bob>
> >>
>
>
>
> -- >
> Claus Ibsen>
> ----------------->
> http://davsclaus.com @davsclaus>
> Camel in Action 2: https://www.manning.com/ibsen2>
>
Re: Content-Length Header being used previous HTTP Response
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yeah we should look at improving this. I have logged a ticket
https://issues.apache.org/jira/browse/CAMEL-12751
You are welcome to help around and provide unit tests / patches etc.
For example as github PRs. We love contributions
http://camel.apache.org/contributing
On Thu, Aug 16, 2018 at 3:29 PM Bob Paulin <bo...@bobpaulin.com> wrote:
>
> Agree that removeHeader inbetween the routes fixes the problem. It's
> great experience having to go through all the code to figure out a
> content-length header mismatch is the issue since the request just
> hangs. I ran into it again with the following scenerio using the
> automatic json binding in the REST DSL with stream caching enabled.
>
> this.getContext().setStreamCaching(true);
>
> restConfiguration().component("netty4-http").port(8282).bindingMode(RestBindingMode.json);
>
> rest("/bind").post()
> .type(InputDTO.class)
> .outType(OutputDTO.class)
> .to("direct:bind");
>
> from("direct:bind")
> .setHeader(Exchange.HTTP_METHOD, constant("POST"))
> .setHeader("Accept", constant("application/json"))
> .setHeader(Exchange.HTTP_PATH, constant(""))
> .setHeader(Exchange.CONTENT_TYPE,
> constant("application/json"))
> .process(new Processor() {
>
> public void process(Exchange exchange) throws
> Exception {
> InputDTO body =
> exchange.getIn().getBody(InputDTO.class);
> body.setDescription("SomethingElse");
>
> }
> })
> .marshal().json(JsonLibrary.Jackson)
> .to("http4://localhost/realservice?bridgeEndpoint=true")
> .unmarshal().json(JsonLibrary.Jackson,
> FinalOutputDTO.class);
>
> The json coming into /bind is pretty printed so includes spaces and
> returns. After it's marshaled to an object and the object is modified
> the Content-length will change. Seems like removing the content-length
> header in the DataFormat used to do the marshal ling or perhaps
> providing an option in the HTTP Client to ignore the content-header
> would be useful. Again willing to help collaborate on a patch but I'd
> like some input on where it would fit best. Currently thinking either
> in the DataFormat marshal methods and/or the HttpProducer.
>
> - Bob
>
> On 2017/09/20 13:13:06, Claus Ibsen <c....@gmail.com> wrote:
> > You can remove the header before calling the http endpoints, with>
> > removeHeader(Exchange.CONTENT_LENGTH)>
> >
> > On Sat, Sep 16, 2017 at 9:03 PM, Bob Paulin <bo...@bobpaulin.com> wrote:>
> > > Hi,>
> > >>
> > > I have a camel rest route like the following>
> > >>
> > > rest("/api/test")>
> > > .post()>
> > > .to("direct:test");>
> > >>
> > > from("direct:test").>
> > >>
> > >
> .toD("http4://localhost/test?bridgeEndpoint=true&throwExceptionOnFailure=false")>
>
> > >>
> > >
> .toD("http4://localhost/test2?bridgeEndpoint=true&throwExceptionOnFailure=false");>
>
> > >>
> > >>
> > > Trouble occurs when the http4://localhost/test route sets a>
> > > content-length header it gets used for http4://localhost/test2. There>
> > > are some web servers that use that header to determine request size>
> > > which if it never gets to the needed length will never close and
> after a>
> > > while timeout. This happens despite content-length being in the>
> > > HttpHeaderFilterStrategy list because in the HttpProducer class
> there is>
> > > code like this. So an input stream will still use the header.>
> > >>
> > > // fallback as input stream>
> > > if (answer == null) {>
> > > // force the body as an input stream since this>
> > > is the fallback>
> > > InputStream is =>
> > > in.getMandatoryBody(InputStream.class);>
> > > String length =>
> > > in.getHeader(Exchange.CONTENT_LENGTH, String.class);>
> > > InputStreamEntity entity = null;>
> > > if (ObjectHelper.isEmpty(length) ) {>
> > > entity = new InputStreamEntity(is, -1);>
> > > } else {>
> > > entity = new InputStreamEntity(is,>
> > > Long.parseLong(length));>
> > > }>
> > > if (contentType != null) {>
> > > entity.setContentType(contentType.toString());>
> > > }>
> > > answer = entity;>
> > > }>
> > >>
> > > I'm assuming there are cases where I might want to explicitly provide>
> > > the content-length to the request entity. But I'd like to be able to>
> > > turn it off so it always falls back to -1 (chunked).>
> > >>
> > > I'm working around the issue by doing this:>
> > >>
> > > from("direct:test").>
> > >>
> > >
> .toD("http4://localhost/test?bridgeEndpoint=true&throwExceptionOnFailure=false")>
>
> > >>
> > > .setHeader("Content-Length", constant(""))>
> > >>
> > >
> .toD("http4://localhost/test2?bridgeEndpoint=true&throwExceptionOnFailure=false");>
>
> > >>
> > > But I'd like to suggest creating a more global way to ignore the>
> > > Content-Length header:>
> > >>
> > > // fallback as input stream>
> > > if (answer == null) {>
> > > // force the body as an input stream since this>
> > > is the fallback>
> > > InputStream is =>
> > > in.getMandatoryBody(InputStream.class);>
> > > String length =>
> > > in.getHeader(Exchange.CONTENT_LENGTH, String.class);>
> > > InputStreamEntity entity = null;>
> > > if (ObjectHelper.isEmpty(length) ||>
> > > getEndpoint().getComponent().isForceChunkedRequestEntity()) {>
> > > entity = new InputStreamEntity(is, -1);>
> > > } else {>
> > > entity = new InputStreamEntity(is,>
> > > Long.parseLong(length));>
> > > }>
> > > if (contentType != null) {>
> > > entity.setContentType(contentType.toString());>
> > > }>
> > > answer = entity;>
> > > }>
> > >>
> > >>
> > > Thoughts on this approach? How are other folks dealing with this?>
> > > Happy to work on a contribution/patch but I wanted to discuss before>
> > > creating a jira.>
> > >>
> > >>
> > > - Bob>
> > >>
> >
> >
> >
> > -- >
> > Claus Ibsen>
> > ----------------->
> > http://davsclaus.com @davsclaus>
> > Camel in Action 2: https://www.manning.com/ibsen2>
> >
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Content-Length Header being used previous HTTP Response
Posted by Bob Paulin <bo...@bobpaulin.com>.
Sorry that should read "not a great experience". I was not attempting
sarcasm for effect there. Apologies for any misunderstand and the
extra noise on the list.
- Bob
On 8/16/2018 8:29 AM, Bob Paulin wrote:
> Agree that removeHeader inbetween the routes fixes the problem. It's
> great experience having to go through all the code to figure out a
> content-length header mismatch is the issue since the request just
> hangs. I ran into it again with the following scenerio using the
> automatic json binding in the REST DSL with stream caching enabled.
>
> this.getContext().setStreamCaching(true);
>
> restConfiguration().component("netty4-http").port(8282).bindingMode(RestBindingMode.json);
>
> rest("/bind").post()
> .type(InputDTO.class)
> .outType(OutputDTO.class)
> .to("direct:bind");
>
> from("direct:bind")
> .setHeader(Exchange.HTTP_METHOD, constant("POST"))
> .setHeader("Accept", constant("application/json"))
> .setHeader(Exchange.HTTP_PATH, constant(""))
> .setHeader(Exchange.CONTENT_TYPE,
> constant("application/json"))
> .process(new Processor() {
>
> public void process(Exchange exchange) throws
> Exception {
> InputDTO body =
> exchange.getIn().getBody(InputDTO.class);
> body.setDescription("SomethingElse");
>
> }
> })
> .marshal().json(JsonLibrary.Jackson)
> .to("http4://localhost/realservice?bridgeEndpoint=true")
> .unmarshal().json(JsonLibrary.Jackson,
> FinalOutputDTO.class);
>
> The json coming into /bind is pretty printed so includes spaces and
> returns. After it's marshaled to an object and the object is modified
> the Content-length will change. Seems like removing the content-length
> header in the DataFormat used to do the marshal ling or perhaps
> providing an option in the HTTP Client to ignore the content-header
> would be useful. Again willing to help collaborate on a patch but I'd
> like some input on where it would fit best. Currently thinking either
> in the DataFormat marshal methods and/or the HttpProducer.
>
> - Bob
>
> On 2017/09/20 13:13:06, Claus Ibsen <c....@gmail.com> wrote:
>> You can remove the header before calling the http endpoints, with>
>> removeHeader(Exchange.CONTENT_LENGTH)>
>>
>> On Sat, Sep 16, 2017 at 9:03 PM, Bob Paulin <bo...@bobpaulin.com> wrote:>
>>> Hi,>
>>> I have a camel rest route like the following>
>>> rest("/api/test")>
>>> .post()>
>>> .to("direct:test");>
>>> from("direct:test").>
> .toD("http4://localhost/test?bridgeEndpoint=true&throwExceptionOnFailure=false")>
>
> .toD("http4://localhost/test2?bridgeEndpoint=true&throwExceptionOnFailure=false");>
>
>>>>
>>> Trouble occurs when the http4://localhost/test route sets a>
>>> content-length header it gets used for http4://localhost/test2. There>
>>> are some web servers that use that header to determine request size>
>>> which if it never gets to the needed length will never close and
> after a>
>>> while timeout. This happens despite content-length being in the>
>>> HttpHeaderFilterStrategy list because in the HttpProducer class
> there is>
>>> code like this. So an input stream will still use the header.>
>>> // fallback as input stream>
>>> if (answer == null) {>
>>> // force the body as an input stream since this>
>>> is the fallback>
>>> InputStream is =>
>>> in.getMandatoryBody(InputStream.class);>
>>> String length =>
>>> in.getHeader(Exchange.CONTENT_LENGTH, String.class);>
>>> InputStreamEntity entity = null;>
>>> if (ObjectHelper.isEmpty(length) ) {>
>>> entity = new InputStreamEntity(is, -1);>
>>> } else {>
>>> entity = new InputStreamEntity(is,>
>>> Long.parseLong(length));>
>>> }>
>>> if (contentType != null) {>
>>> entity.setContentType(contentType.toString());>
>>> }>
>>> answer = entity;>
>>> }>
>>> I'm assuming there are cases where I might want to explicitly provide>
>>> the content-length to the request entity. But I'd like to be able to>
>>> turn it off so it always falls back to -1 (chunked).>
>>> I'm working around the issue by doing this:>
>>> from("direct:test").>
> .toD("http4://localhost/test?bridgeEndpoint=true&throwExceptionOnFailure=false")>
>
>>> .setHeader("Content-Length", constant(""))>
> .toD("http4://localhost/test2?bridgeEndpoint=true&throwExceptionOnFailure=false");>
>
>>> But I'd like to suggest creating a more global way to ignore the>
>>> Content-Length header:>
>>> // fallback as input stream>
>>> if (answer == null) {>
>>> // force the body as an input stream since this>
>>> is the fallback>
>>> InputStream is =>
>>> in.getMandatoryBody(InputStream.class);>
>>> String length =>
>>> in.getHeader(Exchange.CONTENT_LENGTH, String.class);>
>>> InputStreamEntity entity = null;>
>>> if (ObjectHelper.isEmpty(length) ||>
>>> getEndpoint().getComponent().isForceChunkedRequestEntity()) {>
>>> entity = new InputStreamEntity(is, -1);>
>>> } else {>
>>> entity = new InputStreamEntity(is,>
>>> Long.parseLong(length));>
>>> }>
>>> if (contentType != null) {>
>>> entity.setContentType(contentType.toString());>
>>> }>
>>> answer = entity;>
>>> }>
>>>>
>>> Thoughts on this approach? How are other folks dealing with this?>
>>> Happy to work on a contribution/patch but I wanted to discuss before>
>>> creating a jira.>
>>>>
>>> - Bob>
>>
>>
>> -- >
>> Claus Ibsen>
>> ----------------->
>> http://davsclaus.com @davsclaus>
>> Camel in Action 2: https://www.manning.com/ibsen2>
>>