You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "Oleg.Kasian" <o-...@yandex.ru> on 2012/10/19 09:24:53 UTC
Problems when forwarding large post with jetty
I am facing the problem when forwarding post requests with large data using
camel-jetty component. I am not sure what is the reason, stack trace is:
2012-10-19 10:04:30.991:WARN:oejs.HttpConnection:header full:
java.lang.RuntimeException: Header>6144
2012-10-19 10:04:30.992:WARN:oejs.Response:Committed before 500 null
2012-10-19 10:04:30.992:WARN:oejs.HttpConnection:/forward
java.lang.IllegalStateException: Committed
at org.eclipse.jetty.server.Response.resetBuffer(Response.java:1059)
...
camel version 2.10.0
jetty version 7.5.4.v20111024
The problem first occured when running camel based app on karaf 2.2.9,
however I can reproduce it with this unit test:
import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.util.Random;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.http.HttpStatus;
import org.junit.Test;
public class JettyPostBigDataTest extends CamelTestSupport {
private static final int LENGTH = 5 * 1024;
@Test
public void testPostParameter() throws Exception {
final String body = generateBody(LENGTH);
final NameValuePair[] data = {
new NameValuePair("param", body)
};
final HttpClient client = new HttpClient();
final PostMethod post = new
PostMethod("http://localhost:5757/test");
post.setRequestBody(data);
client.executeMethod(post);
final InputStream response = post.getResponseBodyAsStream();
final String out =
context.getTypeConverter().convertTo(String.class, response);
assertEquals("ERROR OCCURED: \n" + out, HttpStatus.SC_OK,
post.getStatusCode());
}
private String generateBody(final int length) throws
UnsupportedEncodingException {
final Random rnd = new Random();
byte[] bytes = new byte[length];
rnd.nextBytes(bytes);
return new String(bytes);
}
@Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
public void configure() {
from("jetty:http://localhost:5757/test").to("http://localhost:5757/forward?bridgeEndpoint=true");
from("jetty:http://localhost:5757/forward")
.setBody(simple("${in.header.param[0]}"))
.convertBodyTo(String.class)
.to("mock:result");
}
};
}
}
--
View this message in context: http://camel.465427.n5.nabble.com/Problems-when-forwarding-large-post-with-jetty-tp5721278.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problems when forwarding large post with jetty
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Oct 19, 2012 at 10:50 AM, Oleg.Kasian <o-...@yandex.ru> wrote:
> It is actually in body, it is application/x-www-form-urlencoded. Seems like,
> when jetty component receives such request, it parses request body to
> exchange headers, passing them to endpoint. In this case, simple forwarding
> like this:
>
> <from uri="jetty://http://0.0.0.0:7777/ws/predmine/controls" />
> <to uri="http://10.1.193.197:8080/predmine/boards.xml?bridgeEndpoint=true"
> />
>
> fails on large data. This doesn't work for requests sent directly from
> browsers either.
> Thanks to Raul, I was able to overcome the problem modifying my route:
>
> <from uri="jetty://http://0.0.0.0:7777/ws/predmine/controls" />
> <removeHeaders pattern="*" />
> <setHeader headerName="content-type">
> <simple>application/x-www-form-urlencoded</simple>
> </setHeader>
> <to uri="http://10.1.193.197:8080/predmine/boards.xml?bridgeEndpoint=true"
> />
>
Ah seems like we should add some logic to support this use-case. As
the application/x-www-form-urlencoded content-type should IMHO be
parsed straight to the message body.
Fell free to log a JIRA ticket.
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Problems-when-forwarding-large-post-with-jetty-tp5721278p5721286.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Problems when forwarding large post with jetty
Posted by "Oleg.Kasian" <o-...@yandex.ru>.
It is actually in body, it is application/x-www-form-urlencoded. Seems like,
when jetty component receives such request, it parses request body to
exchange headers, passing them to endpoint. In this case, simple forwarding
like this:
<from uri="jetty://http://0.0.0.0:7777/ws/predmine/controls" />
<to uri="http://10.1.193.197:8080/predmine/boards.xml?bridgeEndpoint=true"
/>
fails on large data. This doesn't work for requests sent directly from
browsers either.
Thanks to Raul, I was able to overcome the problem modifying my route:
<from uri="jetty://http://0.0.0.0:7777/ws/predmine/controls" />
<removeHeaders pattern="*" />
<setHeader headerName="content-type">
<simple>application/x-www-form-urlencoded</simple>
</setHeader>
<to uri="http://10.1.193.197:8080/predmine/boards.xml?bridgeEndpoint=true"
/>
--
View this message in context: http://camel.465427.n5.nabble.com/Problems-when-forwarding-large-post-with-jetty-tp5721278p5721286.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problems when forwarding large post with jetty
Posted by Willem jiang <wi...@gmail.com>.
Yes, it is not a good practice to send the big message in the http request header.
Why don't put it into the http request body?
--
Willem Jiang
Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) (English)
http://jnn.javaeye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: willemjiang
On Friday, October 19, 2012 at 3:34 PM, Raul Kripalani wrote:
> By default, Camel will propagate all message headers as HTTP headers to the
> client when the response is sent back.
>
> Jetty doesn't like headers larger than 6kb - for a good reason, as it's not
> a good practice to send such large headers. All Web servers have some
> limitation in this respect I think.
>
> In the first route, just after you send to the second one, use a
> removeHeaders() DSL to remove the 'param' header, and you should be okay.
>
> If you want a more automatic approach, you can consider setting exclusion
> patterns in the Header Filter Strategy.
>
> Regards.
>
> Sent from a mobile device
> On 19 Oct 2012 09:25, "Oleg.Kasian" <o-kasian@yandex.ru (mailto:o-kasian@yandex.ru)> wrote:
>
> > I am facing the problem when forwarding post requests with large data using
> > camel-jetty component. I am not sure what is the reason, stack trace is:
> >
> > 2012-10-19 10:04:30.991:WARN:oejs.HttpConnection:header full:
> > java.lang.RuntimeException: Header>6144
> > 2012-10-19 10:04:30.992:WARN:oejs.Response:Committed before 500 null
> > 2012-10-19 10:04:30.992:WARN:oejs.HttpConnection:/forward
> > java.lang.IllegalStateException: Committed
> > at
> > org.eclipse.jetty.server.Response.resetBuffer(Response.java:1059)
> > ...
> >
> > camel version 2.10.0
> > jetty version 7.5.4.v20111024
> > The problem first occured when running camel based app on karaf 2.2.9,
> > however I can reproduce it with this unit test:
> >
> > import java.io.InputStream;
> > import java.io.UnsupportedEncodingException;
> > import java.util.Random;
> > import org.apache.camel.builder.RouteBuilder;
> > import org.apache.camel.test.junit4.CamelTestSupport;
> > import org.apache.commons.httpclient.HttpClient;
> > import org.apache.commons.httpclient.NameValuePair;
> > import org.apache.commons.httpclient.methods.PostMethod;
> > import org.apache.http.HttpStatus;
> > import org.junit.Test;
> >
> >
> > public class JettyPostBigDataTest extends CamelTestSupport {
> >
> > private static final int LENGTH = 5 * 1024;
> >
> > @Test
> > public void testPostParameter() throws Exception {
> > final String body = generateBody(LENGTH);
> > final NameValuePair[] data = {
> > new NameValuePair("param", body)
> > };
> > final HttpClient client = new HttpClient();
> > final PostMethod post = new
> > PostMethod("http://localhost:5757/test");
> > post.setRequestBody(data);
> > client.executeMethod(post);
> > final InputStream response = post.getResponseBodyAsStream();
> > final String out =
> > context.getTypeConverter().convertTo(String.class, response);
> > assertEquals("ERROR OCCURED: \n" + out, HttpStatus.SC_OK,
> > post.getStatusCode());
> > }
> >
> > private String generateBody(final int length) throws
> > UnsupportedEncodingException {
> > final Random rnd = new Random();
> > byte[] bytes = new byte[length];
> > rnd.nextBytes(bytes);
> > return new String(bytes);
> > }
> >
> > @Override
> > protected RouteBuilder createRouteBuilder() throws Exception {
> > return new RouteBuilder() {
> > public void configure() {
> >
> >
> > from("jetty:http://localhost:5757/test").to("
> > http://localhost:5757/forward?bridgeEndpoint=true");
> >
> > from("jetty:http://localhost:5757/forward")
> > .setBody(simple("${in.header.param[0]}"))
> > .convertBodyTo(String.class)
> > .to("mock:result");
> >
> > }
> > };
> > }
> > }
> >
> >
> >
> > --
> > View this message in context:
> > http://camel.465427.n5.nabble.com/Problems-when-forwarding-large-post-with-jetty-tp5721278.html
> > Sent from the Camel - Users mailing list archive at Nabble.com (http://Nabble.com).
>
Re: Problems when forwarding large post with jetty
Posted by Raul Kripalani <ra...@evosent.com>.
By default, Camel will propagate all message headers as HTTP headers to the
client when the response is sent back.
Jetty doesn't like headers larger than 6kb - for a good reason, as it's not
a good practice to send such large headers. All Web servers have some
limitation in this respect I think.
In the first route, just after you send to the second one, use a
removeHeaders() DSL to remove the 'param' header, and you should be okay.
If you want a more automatic approach, you can consider setting exclusion
patterns in the Header Filter Strategy.
Regards.
Sent from a mobile device
On 19 Oct 2012 09:25, "Oleg.Kasian" <o-...@yandex.ru> wrote:
> I am facing the problem when forwarding post requests with large data using
> camel-jetty component. I am not sure what is the reason, stack trace is:
>
> 2012-10-19 10:04:30.991:WARN:oejs.HttpConnection:header full:
> java.lang.RuntimeException: Header>6144
> 2012-10-19 10:04:30.992:WARN:oejs.Response:Committed before 500 null
> 2012-10-19 10:04:30.992:WARN:oejs.HttpConnection:/forward
> java.lang.IllegalStateException: Committed
> at
> org.eclipse.jetty.server.Response.resetBuffer(Response.java:1059)
> ...
>
> camel version 2.10.0
> jetty version 7.5.4.v20111024
> The problem first occured when running camel based app on karaf 2.2.9,
> however I can reproduce it with this unit test:
>
> import java.io.InputStream;
> import java.io.UnsupportedEncodingException;
> import java.util.Random;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.apache.commons.httpclient.HttpClient;
> import org.apache.commons.httpclient.NameValuePair;
> import org.apache.commons.httpclient.methods.PostMethod;
> import org.apache.http.HttpStatus;
> import org.junit.Test;
>
>
> public class JettyPostBigDataTest extends CamelTestSupport {
>
> private static final int LENGTH = 5 * 1024;
>
> @Test
> public void testPostParameter() throws Exception {
> final String body = generateBody(LENGTH);
> final NameValuePair[] data = {
> new NameValuePair("param", body)
> };
> final HttpClient client = new HttpClient();
> final PostMethod post = new
> PostMethod("http://localhost:5757/test");
> post.setRequestBody(data);
> client.executeMethod(post);
> final InputStream response = post.getResponseBodyAsStream();
> final String out =
> context.getTypeConverter().convertTo(String.class, response);
> assertEquals("ERROR OCCURED: \n" + out, HttpStatus.SC_OK,
> post.getStatusCode());
> }
>
> private String generateBody(final int length) throws
> UnsupportedEncodingException {
> final Random rnd = new Random();
> byte[] bytes = new byte[length];
> rnd.nextBytes(bytes);
> return new String(bytes);
> }
>
> @Override
> protected RouteBuilder createRouteBuilder() throws Exception {
> return new RouteBuilder() {
> public void configure() {
>
>
> from("jetty:http://localhost:5757/test").to("
> http://localhost:5757/forward?bridgeEndpoint=true");
>
> from("jetty:http://localhost:5757/forward")
> .setBody(simple("${in.header.param[0]}"))
> .convertBodyTo(String.class)
> .to("mock:result");
>
> }
> };
> }
> }
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Problems-when-forwarding-large-post-with-jetty-tp5721278.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>