You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by _Jens <je...@icw.de> on 2010/05/17 14:52:50 UTC
2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for
Camel-2686 and CAMEL-2694 (rev 941275)?
Hi,
it seems to me that the DefaultHTTPBinding again uses a CachedOutputStream.
This means that the binding cannot be used anymore with longer streams and
that the streamCache option on the CamelContext bean is ignored. Is this
going to be the way the caching is done in 2.3?
Thanks,
Jens
--
View this message in context: http://old.nabble.com/2.3-SNAPSHOT%3A-Stream-Caching-again-enabled-by-Bug-Fix-for-Camel-2686-and-CAMEL-2694-%28rev-941275%29--tp28582952p28582952.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for Camel-2686
and CAMEL-2694 (rev 941275)?
Posted by Willem Jiang <wi...@gmail.com>.
Hi Jens,
It's my bad, I should not try to covert the message into StreamCache.
As it will consume the input stream, I will commit a quick fix shortly.
Willem
_Jens wrote:
> Hi Willem,
>
> unfortunately I was not able to get this working. I saw the unit test that
> you checked in for this. The test runs without an error. However, it does
> not verify this completely. Although it is now possible to get the stream,
> the stream has been read. I changed the route builder used in the test to
> this:
>
>
> from("jetty:http://localhost:9083/noStreamCache?disableStreamCache=true").noStreamCaching().process(new
> Processor() {
>
> public void process(Exchange exchange) throws Exception
> {
> InputStream is =
> (InputStream)exchange.getIn().getBody();
> assertTrue(is.read() != -1);
> System.out.println(is.getClass());
> assertTrue("It should be a raw inputstream", is
> instanceof org.eclipse.jetty.server.HttpInput);
> exchange.getOut().setBody("OK");
> }
>
> });
>
> And then the check for "is.read() != -1" fails. The reason for this is
> within DefaultHttpBinding.readRequest:
>
> // reset the stream cache
> StreamCache cache = message.getBody(StreamCache.class);
>
> This reads the actual input stream into a CachedInputStream, no matter what
> settings are disabled.
>
> Jens
>
>
>
> willem.jiang wrote:
>> Just FYI,
>>
>> I introduced a disableStreamCache option into HttpComponent[1], by
>> setting this option to be true, you will get the pure input stream that
>> Jetty provides.
>>
>> [1] https://issues.apache.org/activemq/browse/CAMEL-2738
>>
>>
>> Willem
>>
>>
>
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for
Camel-2686 and CAMEL-2694 (rev 941275)?
Posted by _Jens <je...@icw.de>.
Hi Willem,
unfortunately I was not able to get this working. I saw the unit test that
you checked in for this. The test runs without an error. However, it does
not verify this completely. Although it is now possible to get the stream,
the stream has been read. I changed the route builder used in the test to
this:
from("jetty:http://localhost:9083/noStreamCache?disableStreamCache=true").noStreamCaching().process(new
Processor() {
public void process(Exchange exchange) throws Exception
{
InputStream is =
(InputStream)exchange.getIn().getBody();
assertTrue(is.read() != -1);
System.out.println(is.getClass());
assertTrue("It should be a raw inputstream", is
instanceof org.eclipse.jetty.server.HttpInput);
exchange.getOut().setBody("OK");
}
});
And then the check for "is.read() != -1" fails. The reason for this is
within DefaultHttpBinding.readRequest:
// reset the stream cache
StreamCache cache = message.getBody(StreamCache.class);
This reads the actual input stream into a CachedInputStream, no matter what
settings are disabled.
Jens
willem.jiang wrote:
>
> Just FYI,
>
> I introduced a disableStreamCache option into HttpComponent[1], by
> setting this option to be true, you will get the pure input stream that
> Jetty provides.
>
> [1] https://issues.apache.org/activemq/browse/CAMEL-2738
>
>
> Willem
>
>
--
View this message in context: http://old.nabble.com/2.3-SNAPSHOT%3A-Stream-Caching-again-enabled-by-Bug-Fix-for-Camel-2686-and-CAMEL-2694-%28rev-941275%29--tp28582952p28619827.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for Camel-2686
and CAMEL-2694 (rev 941275)?
Posted by Willem Jiang <wi...@gmail.com>.
Just FYI,
I introduced a disableStreamCache option into HttpComponent[1], by
setting this option to be true, you will get the pure input stream that
Jetty provides.
[1] https://issues.apache.org/activemq/browse/CAMEL-2738
Willem
_Jens wrote:
> I would like to disable the feature in the DefaultHttpBinding because of
> existing code. Our code already reads the stream content into a file, so we
> do not require this feature from the Http binding. And obviously we don't
> want to write the data twice. This is why I was asking if this is the
> intended behavior for 2.3. In that case we need to implement an HTTP binding
> that bypasses that feature.
> To be honest, I think that the Jetty component should allow me to get access
> to the pure input stream that Jetty provides. Any additional streaming into
> a file is a nice feature but should be optional. But this is up to you guys.
> I simply need to make sure that we can upgrade to 2.3 as soon as it is
> released.
>
>
> willem.jiang wrote:
>> Yes, from this point, you can't disable the Stream Caching.
>> But there is a difference between the Stream Caching and current Http
>> binding change.
>> If you don't enable the Stream Caching, there is no interceptor will be
>> used to reset the Stream for you before the message is routing to the
>> next endpoint.
>>
>> If there are large Post message body, the Stream Caching will help you
>> avoid the OutOfMessage error as it can dump the message into a file.
>> I don't know why are your going to disable it.
>>
>> Willem
>>
>> _Jens wrote:
>>> Hi Willem,
>>>
>>> thanks for your quick reply. The issue is that I cannot disable
>>> streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
>>> DefaultHttpBinding.parseBody:
>>>
>>> Before:
>>>
>>> // otherwise use input stream
>>> return HttpConverter.toInputStream(request);
>>>
>>> Now:
>>> // otherwise use input stream and we need to cache it first
>>> InputStream is = HttpConverter.toInputStream(request,
>>> httpMessage.getExchange());
>>> if (is == null) {
>>> return is;
>>> }
>>> // convert the input stream to StreamCache
>>> try {
>>> CachedOutputStream cos = new
>>> CachedOutputStream(httpMessage.getExchange());
>>> IOHelper.copy(is, cos);
>>> return cos.getStreamCache();
>>> } finally {
>>> is.close();
>>> }
>>>
>>> This means, the CachedOutputStream is always used. I have tried to
>>> disable
>>> this on the camelContext. But obviously the above code does not check
>>> this
>>> option in the camelContext. So it is not possible anymore.
>>>
>>>
>>>
>>>
>>> willem.jiang wrote:
>>>> These changes just to make sure the we can still read the post message
>>>> body even the body is form message.
>>>>
>>>> You can still configure the CachedOutputStream by using the camelContext
>>>> properties, I don't know what kind of issue that the changes introduced
>>>> to you.
>>>>
>>>> Willem
>>>>
>>>> _Jens wrote:
>>>>> Hi,
>>>>>
>>>>> it seems to me that the DefaultHTTPBinding again uses a
>>>>> CachedOutputStream.
>>>>> This means that the binding cannot be used anymore with longer streams
>>>>> and
>>>>> that the streamCache option on the CamelContext bean is ignored. Is
>>>>> this
>>>>> going to be the way the caching is done in 2.3?
>>>>>
>>>>> Thanks,
>>>>> Jens
>>>>
>>
>>
>
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for
Camel-2686 and CAMEL-2694 (rev 941275)?
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, May 17, 2010 at 4:36 PM, Gert Vanthienen
<ge...@gmail.com> wrote:
> L.S.,
>
> Instead of having this option on the endpoint, wouldn't it make more
> sense to somehow attach that as an attribute/policy to the route or on
> the routebuilder? That way, people can enable/disable stream caching
> on that level instead of having to worry about which endpoints in
> their routes need special flags to enable/disable it.
>
There is already a steamCaching in the DSL. So you can just do
from(jetty).streamCaching().xxx
So I do think that conversion to StreamCache should be removed from
HTTPBinding. Then end users have to specify .streamCaching() in the
route to leverage it. But I suspect people may forget / dont know how
to. And thus willem made that automatic.
> Regards,
>
> Gert Vanthienen
> ------------------------
> Open Source SOA: http://fusesource.com
> Blog: http://gertvanthienen.blogspot.com/
>
>
>
> On 17 May 2010 16:19, Claus Ibsen <cl...@gmail.com> wrote:
>> On Mon, May 17, 2010 at 3:58 PM, _Jens <je...@icw.de> wrote:
>>>
>>> I would like to disable the feature in the DefaultHttpBinding because of
>>> existing code. Our code already reads the stream content into a file, so we
>>> do not require this feature from the Http binding. And obviously we don't
>>> want to write the data twice. This is why I was asking if this is the
>>> intended behavior for 2.3. In that case we need to implement an HTTP binding
>>> that bypasses that feature.
>>> To be honest, I think that the Jetty component should allow me to get access
>>> to the pure input stream that Jetty provides. Any additional streaming into
>>> a file is a nice feature but should be optional. But this is up to you guys.
>>> I simply need to make sure that we can upgrade to 2.3 as soon as it is
>>> released.
>>>
>>
>> I think the end user should *always* have the choice. And the cache
>> stream have bitten us before.
>> So I think willem should create an option which you can enable on the
>> endpoint to tell it to use that cached stream or not.
>>
>>
>>>
>>> willem.jiang wrote:
>>>>
>>>> Yes, from this point, you can't disable the Stream Caching.
>>>> But there is a difference between the Stream Caching and current Http
>>>> binding change.
>>>> If you don't enable the Stream Caching, there is no interceptor will be
>>>> used to reset the Stream for you before the message is routing to the
>>>> next endpoint.
>>>>
>>>> If there are large Post message body, the Stream Caching will help you
>>>> avoid the OutOfMessage error as it can dump the message into a file.
>>>> I don't know why are your going to disable it.
>>>>
>>>> Willem
>>>>
>>>> _Jens wrote:
>>>>> Hi Willem,
>>>>>
>>>>> thanks for your quick reply. The issue is that I cannot disable
>>>>> streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
>>>>> DefaultHttpBinding.parseBody:
>>>>>
>>>>> Before:
>>>>>
>>>>> // otherwise use input stream
>>>>> return HttpConverter.toInputStream(request);
>>>>>
>>>>> Now:
>>>>> // otherwise use input stream and we need to cache it first
>>>>> InputStream is = HttpConverter.toInputStream(request,
>>>>> httpMessage.getExchange());
>>>>> if (is == null) {
>>>>> return is;
>>>>> }
>>>>> // convert the input stream to StreamCache
>>>>> try {
>>>>> CachedOutputStream cos = new
>>>>> CachedOutputStream(httpMessage.getExchange());
>>>>> IOHelper.copy(is, cos);
>>>>> return cos.getStreamCache();
>>>>> } finally {
>>>>> is.close();
>>>>> }
>>>>>
>>>>> This means, the CachedOutputStream is always used. I have tried to
>>>>> disable
>>>>> this on the camelContext. But obviously the above code does not check
>>>>> this
>>>>> option in the camelContext. So it is not possible anymore.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> willem.jiang wrote:
>>>>>> These changes just to make sure the we can still read the post message
>>>>>> body even the body is form message.
>>>>>>
>>>>>> You can still configure the CachedOutputStream by using the camelContext
>>>>>> properties, I don't know what kind of issue that the changes introduced
>>>>>> to you.
>>>>>>
>>>>>> Willem
>>>>>>
>>>>>> _Jens wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> it seems to me that the DefaultHTTPBinding again uses a
>>>>>>> CachedOutputStream.
>>>>>>> This means that the binding cannot be used anymore with longer streams
>>>>>>> and
>>>>>>> that the streamCache option on the CamelContext bean is ignored. Is
>>>>>>> this
>>>>>>> going to be the way the caching is done in 2.3?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Jens
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context: http://old.nabble.com/2.3-SNAPSHOT%3A-Stream-Caching-again-enabled-by-Bug-Fix-for-Camel-2686-and-CAMEL-2694-%28rev-941275%29--tp28582952p28583757.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for
Camel-2686 and CAMEL-2694 (rev 941275)?
Posted by Gert Vanthienen <ge...@gmail.com>.
L.S.,
Instead of having this option on the endpoint, wouldn't it make more
sense to somehow attach that as an attribute/policy to the route or on
the routebuilder? That way, people can enable/disable stream caching
on that level instead of having to worry about which endpoints in
their routes need special flags to enable/disable it.
Regards,
Gert Vanthienen
------------------------
Open Source SOA: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/
On 17 May 2010 16:19, Claus Ibsen <cl...@gmail.com> wrote:
> On Mon, May 17, 2010 at 3:58 PM, _Jens <je...@icw.de> wrote:
>>
>> I would like to disable the feature in the DefaultHttpBinding because of
>> existing code. Our code already reads the stream content into a file, so we
>> do not require this feature from the Http binding. And obviously we don't
>> want to write the data twice. This is why I was asking if this is the
>> intended behavior for 2.3. In that case we need to implement an HTTP binding
>> that bypasses that feature.
>> To be honest, I think that the Jetty component should allow me to get access
>> to the pure input stream that Jetty provides. Any additional streaming into
>> a file is a nice feature but should be optional. But this is up to you guys.
>> I simply need to make sure that we can upgrade to 2.3 as soon as it is
>> released.
>>
>
> I think the end user should *always* have the choice. And the cache
> stream have bitten us before.
> So I think willem should create an option which you can enable on the
> endpoint to tell it to use that cached stream or not.
>
>
>>
>> willem.jiang wrote:
>>>
>>> Yes, from this point, you can't disable the Stream Caching.
>>> But there is a difference between the Stream Caching and current Http
>>> binding change.
>>> If you don't enable the Stream Caching, there is no interceptor will be
>>> used to reset the Stream for you before the message is routing to the
>>> next endpoint.
>>>
>>> If there are large Post message body, the Stream Caching will help you
>>> avoid the OutOfMessage error as it can dump the message into a file.
>>> I don't know why are your going to disable it.
>>>
>>> Willem
>>>
>>> _Jens wrote:
>>>> Hi Willem,
>>>>
>>>> thanks for your quick reply. The issue is that I cannot disable
>>>> streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
>>>> DefaultHttpBinding.parseBody:
>>>>
>>>> Before:
>>>>
>>>> // otherwise use input stream
>>>> return HttpConverter.toInputStream(request);
>>>>
>>>> Now:
>>>> // otherwise use input stream and we need to cache it first
>>>> InputStream is = HttpConverter.toInputStream(request,
>>>> httpMessage.getExchange());
>>>> if (is == null) {
>>>> return is;
>>>> }
>>>> // convert the input stream to StreamCache
>>>> try {
>>>> CachedOutputStream cos = new
>>>> CachedOutputStream(httpMessage.getExchange());
>>>> IOHelper.copy(is, cos);
>>>> return cos.getStreamCache();
>>>> } finally {
>>>> is.close();
>>>> }
>>>>
>>>> This means, the CachedOutputStream is always used. I have tried to
>>>> disable
>>>> this on the camelContext. But obviously the above code does not check
>>>> this
>>>> option in the camelContext. So it is not possible anymore.
>>>>
>>>>
>>>>
>>>>
>>>> willem.jiang wrote:
>>>>> These changes just to make sure the we can still read the post message
>>>>> body even the body is form message.
>>>>>
>>>>> You can still configure the CachedOutputStream by using the camelContext
>>>>> properties, I don't know what kind of issue that the changes introduced
>>>>> to you.
>>>>>
>>>>> Willem
>>>>>
>>>>> _Jens wrote:
>>>>>> Hi,
>>>>>>
>>>>>> it seems to me that the DefaultHTTPBinding again uses a
>>>>>> CachedOutputStream.
>>>>>> This means that the binding cannot be used anymore with longer streams
>>>>>> and
>>>>>> that the streamCache option on the CamelContext bean is ignored. Is
>>>>>> this
>>>>>> going to be the way the caching is done in 2.3?
>>>>>>
>>>>>> Thanks,
>>>>>> Jens
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>> --
>> View this message in context: http://old.nabble.com/2.3-SNAPSHOT%3A-Stream-Caching-again-enabled-by-Bug-Fix-for-Camel-2686-and-CAMEL-2694-%28rev-941275%29--tp28582952p28583757.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for
Camel-2686 and CAMEL-2694 (rev 941275)?
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, May 17, 2010 at 3:58 PM, _Jens <je...@icw.de> wrote:
>
> I would like to disable the feature in the DefaultHttpBinding because of
> existing code. Our code already reads the stream content into a file, so we
> do not require this feature from the Http binding. And obviously we don't
> want to write the data twice. This is why I was asking if this is the
> intended behavior for 2.3. In that case we need to implement an HTTP binding
> that bypasses that feature.
> To be honest, I think that the Jetty component should allow me to get access
> to the pure input stream that Jetty provides. Any additional streaming into
> a file is a nice feature but should be optional. But this is up to you guys.
> I simply need to make sure that we can upgrade to 2.3 as soon as it is
> released.
>
I think the end user should *always* have the choice. And the cache
stream have bitten us before.
So I think willem should create an option which you can enable on the
endpoint to tell it to use that cached stream or not.
>
> willem.jiang wrote:
>>
>> Yes, from this point, you can't disable the Stream Caching.
>> But there is a difference between the Stream Caching and current Http
>> binding change.
>> If you don't enable the Stream Caching, there is no interceptor will be
>> used to reset the Stream for you before the message is routing to the
>> next endpoint.
>>
>> If there are large Post message body, the Stream Caching will help you
>> avoid the OutOfMessage error as it can dump the message into a file.
>> I don't know why are your going to disable it.
>>
>> Willem
>>
>> _Jens wrote:
>>> Hi Willem,
>>>
>>> thanks for your quick reply. The issue is that I cannot disable
>>> streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
>>> DefaultHttpBinding.parseBody:
>>>
>>> Before:
>>>
>>> // otherwise use input stream
>>> return HttpConverter.toInputStream(request);
>>>
>>> Now:
>>> // otherwise use input stream and we need to cache it first
>>> InputStream is = HttpConverter.toInputStream(request,
>>> httpMessage.getExchange());
>>> if (is == null) {
>>> return is;
>>> }
>>> // convert the input stream to StreamCache
>>> try {
>>> CachedOutputStream cos = new
>>> CachedOutputStream(httpMessage.getExchange());
>>> IOHelper.copy(is, cos);
>>> return cos.getStreamCache();
>>> } finally {
>>> is.close();
>>> }
>>>
>>> This means, the CachedOutputStream is always used. I have tried to
>>> disable
>>> this on the camelContext. But obviously the above code does not check
>>> this
>>> option in the camelContext. So it is not possible anymore.
>>>
>>>
>>>
>>>
>>> willem.jiang wrote:
>>>> These changes just to make sure the we can still read the post message
>>>> body even the body is form message.
>>>>
>>>> You can still configure the CachedOutputStream by using the camelContext
>>>> properties, I don't know what kind of issue that the changes introduced
>>>> to you.
>>>>
>>>> Willem
>>>>
>>>> _Jens wrote:
>>>>> Hi,
>>>>>
>>>>> it seems to me that the DefaultHTTPBinding again uses a
>>>>> CachedOutputStream.
>>>>> This means that the binding cannot be used anymore with longer streams
>>>>> and
>>>>> that the streamCache option on the CamelContext bean is ignored. Is
>>>>> this
>>>>> going to be the way the caching is done in 2.3?
>>>>>
>>>>> Thanks,
>>>>> Jens
>>>>
>>>>
>>>
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/2.3-SNAPSHOT%3A-Stream-Caching-again-enabled-by-Bug-Fix-for-Camel-2686-and-CAMEL-2694-%28rev-941275%29--tp28582952p28583757.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for
Camel-2686 and CAMEL-2694 (rev 941275)?
Posted by _Jens <je...@icw.de>.
I would like to disable the feature in the DefaultHttpBinding because of
existing code. Our code already reads the stream content into a file, so we
do not require this feature from the Http binding. And obviously we don't
want to write the data twice. This is why I was asking if this is the
intended behavior for 2.3. In that case we need to implement an HTTP binding
that bypasses that feature.
To be honest, I think that the Jetty component should allow me to get access
to the pure input stream that Jetty provides. Any additional streaming into
a file is a nice feature but should be optional. But this is up to you guys.
I simply need to make sure that we can upgrade to 2.3 as soon as it is
released.
willem.jiang wrote:
>
> Yes, from this point, you can't disable the Stream Caching.
> But there is a difference between the Stream Caching and current Http
> binding change.
> If you don't enable the Stream Caching, there is no interceptor will be
> used to reset the Stream for you before the message is routing to the
> next endpoint.
>
> If there are large Post message body, the Stream Caching will help you
> avoid the OutOfMessage error as it can dump the message into a file.
> I don't know why are your going to disable it.
>
> Willem
>
> _Jens wrote:
>> Hi Willem,
>>
>> thanks for your quick reply. The issue is that I cannot disable
>> streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
>> DefaultHttpBinding.parseBody:
>>
>> Before:
>>
>> // otherwise use input stream
>> return HttpConverter.toInputStream(request);
>>
>> Now:
>> // otherwise use input stream and we need to cache it first
>> InputStream is = HttpConverter.toInputStream(request,
>> httpMessage.getExchange());
>> if (is == null) {
>> return is;
>> }
>> // convert the input stream to StreamCache
>> try {
>> CachedOutputStream cos = new
>> CachedOutputStream(httpMessage.getExchange());
>> IOHelper.copy(is, cos);
>> return cos.getStreamCache();
>> } finally {
>> is.close();
>> }
>>
>> This means, the CachedOutputStream is always used. I have tried to
>> disable
>> this on the camelContext. But obviously the above code does not check
>> this
>> option in the camelContext. So it is not possible anymore.
>>
>>
>>
>>
>> willem.jiang wrote:
>>> These changes just to make sure the we can still read the post message
>>> body even the body is form message.
>>>
>>> You can still configure the CachedOutputStream by using the camelContext
>>> properties, I don't know what kind of issue that the changes introduced
>>> to you.
>>>
>>> Willem
>>>
>>> _Jens wrote:
>>>> Hi,
>>>>
>>>> it seems to me that the DefaultHTTPBinding again uses a
>>>> CachedOutputStream.
>>>> This means that the binding cannot be used anymore with longer streams
>>>> and
>>>> that the streamCache option on the CamelContext bean is ignored. Is
>>>> this
>>>> going to be the way the caching is done in 2.3?
>>>>
>>>> Thanks,
>>>> Jens
>>>
>>>
>>
>
>
>
--
View this message in context: http://old.nabble.com/2.3-SNAPSHOT%3A-Stream-Caching-again-enabled-by-Bug-Fix-for-Camel-2686-and-CAMEL-2694-%28rev-941275%29--tp28582952p28583757.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for Camel-2686
and CAMEL-2694 (rev 941275)?
Posted by Willem Jiang <wi...@gmail.com>.
Claus Ibsen wrote:
> On Mon, May 17, 2010 at 3:29 PM, Willem Jiang <wi...@gmail.com> wrote:
>> Yes, from this point, you can't disable the Stream Caching.
>> But there is a difference between the Stream Caching and current Http
>> binding change.
>> If you don't enable the Stream Caching, there is no interceptor will be
>> used to reset the Stream for you before the message is routing to the next
>> endpoint.
>>
>
>> If there are large Post message body, the Stream Caching will help you avoid
>> the OutOfMessage error as it can dump the message into a file.
>> I don't know why are your going to disable it.
>>
>
> Do you mean OutOfMemoryError ?
>
> I do not think Camel should impose automatic stream cache in these situations.
> The end user should always have the choice.
>
> He could just have enabled stream caching on the route in those situations.
> from(jetty).streamCaching().to(yyy);
>
>
> If there is another reason why you cache the stream in HttpBinding?
>
> There was a link in one of the JIRA tickets to this one:
> http://old.nabble.com/jetty%3Ahttp-component-%22mangles%22-x-www-form-urlencoded-POST-in-message%3A-ts28348176.html
> Which seems to been needed to fix some POST form issue?
> Is this the reason?
>
Yes, this is the reason why I changed the code of DefaultHttpBinding.
I agree we can provide a option for the user who can get the raw
InputStream if they want, but they can't get the POST from from Message
body any more with this option.
Willem
>
>
>
>
>> Willem
>>
>> _Jens wrote:
>>> Hi Willem,
>>>
>>> thanks for your quick reply. The issue is that I cannot disable
>>> streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
>>> DefaultHttpBinding.parseBody:
>>>
>>> Before:
>>>
>>> // otherwise use input stream
>>> return HttpConverter.toInputStream(request);
>>> Now:
>>> // otherwise use input stream and we need to cache it first
>>> InputStream is = HttpConverter.toInputStream(request,
>>> httpMessage.getExchange());
>>> if (is == null) {
>>> return is;
>>> }
>>> // convert the input stream to StreamCache
>>> try {
>>> CachedOutputStream cos = new
>>> CachedOutputStream(httpMessage.getExchange());
>>> IOHelper.copy(is, cos);
>>> return cos.getStreamCache();
>>> } finally {
>>> is.close();
>>> }
>>>
>>> This means, the CachedOutputStream is always used. I have tried to disable
>>> this on the camelContext. But obviously the above code does not check this
>>> option in the camelContext. So it is not possible anymore.
>>>
>>>
>>>
>>>
>>> willem.jiang wrote:
>>>> These changes just to make sure the we can still read the post message
>>>> body even the body is form message.
>>>>
>>>> You can still configure the CachedOutputStream by using the camelContext
>>>> properties, I don't know what kind of issue that the changes introduced to
>>>> you.
>>>>
>>>> Willem
>>>>
>>>> _Jens wrote:
>>>>> Hi,
>>>>>
>>>>> it seems to me that the DefaultHTTPBinding again uses a
>>>>> CachedOutputStream.
>>>>> This means that the binding cannot be used anymore with longer streams
>>>>> and
>>>>> that the streamCache option on the CamelContext bean is ignored. Is this
>>>>> going to be the way the caching is done in 2.3?
>>>>>
>>>>> Thanks,
>>>>> Jens
>>>>
>>
>
>
>
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for
Camel-2686 and CAMEL-2694 (rev 941275)?
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, May 17, 2010 at 3:29 PM, Willem Jiang <wi...@gmail.com> wrote:
> Yes, from this point, you can't disable the Stream Caching.
> But there is a difference between the Stream Caching and current Http
> binding change.
> If you don't enable the Stream Caching, there is no interceptor will be
> used to reset the Stream for you before the message is routing to the next
> endpoint.
>
> If there are large Post message body, the Stream Caching will help you avoid
> the OutOfMessage error as it can dump the message into a file.
> I don't know why are your going to disable it.
>
Do you mean OutOfMemoryError ?
I do not think Camel should impose automatic stream cache in these situations.
The end user should always have the choice.
He could just have enabled stream caching on the route in those situations.
from(jetty).streamCaching().to(yyy);
If there is another reason why you cache the stream in HttpBinding?
There was a link in one of the JIRA tickets to this one:
http://old.nabble.com/jetty%3Ahttp-component-%22mangles%22-x-www-form-urlencoded-POST-in-message%3A-ts28348176.html
Which seems to been needed to fix some POST form issue?
Is this the reason?
> Willem
>
> _Jens wrote:
>>
>> Hi Willem,
>>
>> thanks for your quick reply. The issue is that I cannot disable
>> streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
>> DefaultHttpBinding.parseBody:
>>
>> Before:
>>
>> // otherwise use input stream
>> return HttpConverter.toInputStream(request);
>> Now:
>> // otherwise use input stream and we need to cache it first
>> InputStream is = HttpConverter.toInputStream(request,
>> httpMessage.getExchange());
>> if (is == null) {
>> return is;
>> }
>> // convert the input stream to StreamCache
>> try {
>> CachedOutputStream cos = new
>> CachedOutputStream(httpMessage.getExchange());
>> IOHelper.copy(is, cos);
>> return cos.getStreamCache();
>> } finally {
>> is.close();
>> }
>>
>> This means, the CachedOutputStream is always used. I have tried to disable
>> this on the camelContext. But obviously the above code does not check this
>> option in the camelContext. So it is not possible anymore.
>>
>>
>>
>>
>> willem.jiang wrote:
>>>
>>> These changes just to make sure the we can still read the post message
>>> body even the body is form message.
>>>
>>> You can still configure the CachedOutputStream by using the camelContext
>>> properties, I don't know what kind of issue that the changes introduced to
>>> you.
>>>
>>> Willem
>>>
>>> _Jens wrote:
>>>>
>>>> Hi,
>>>>
>>>> it seems to me that the DefaultHTTPBinding again uses a
>>>> CachedOutputStream.
>>>> This means that the binding cannot be used anymore with longer streams
>>>> and
>>>> that the streamCache option on the CamelContext bean is ignored. Is this
>>>> going to be the way the caching is done in 2.3?
>>>>
>>>> Thanks,
>>>> Jens
>>>
>>>
>>
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for Camel-2686
and CAMEL-2694 (rev 941275)?
Posted by Willem Jiang <wi...@gmail.com>.
Yes, from this point, you can't disable the Stream Caching.
But there is a difference between the Stream Caching and current Http
binding change.
If you don't enable the Stream Caching, there is no interceptor will be
used to reset the Stream for you before the message is routing to the
next endpoint.
If there are large Post message body, the Stream Caching will help you
avoid the OutOfMessage error as it can dump the message into a file.
I don't know why are your going to disable it.
Willem
_Jens wrote:
> Hi Willem,
>
> thanks for your quick reply. The issue is that I cannot disable
> streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
> DefaultHttpBinding.parseBody:
>
> Before:
>
> // otherwise use input stream
> return HttpConverter.toInputStream(request);
>
> Now:
> // otherwise use input stream and we need to cache it first
> InputStream is = HttpConverter.toInputStream(request,
> httpMessage.getExchange());
> if (is == null) {
> return is;
> }
> // convert the input stream to StreamCache
> try {
> CachedOutputStream cos = new
> CachedOutputStream(httpMessage.getExchange());
> IOHelper.copy(is, cos);
> return cos.getStreamCache();
> } finally {
> is.close();
> }
>
> This means, the CachedOutputStream is always used. I have tried to disable
> this on the camelContext. But obviously the above code does not check this
> option in the camelContext. So it is not possible anymore.
>
>
>
>
> willem.jiang wrote:
>> These changes just to make sure the we can still read the post message
>> body even the body is form message.
>>
>> You can still configure the CachedOutputStream by using the camelContext
>> properties, I don't know what kind of issue that the changes introduced
>> to you.
>>
>> Willem
>>
>> _Jens wrote:
>>> Hi,
>>>
>>> it seems to me that the DefaultHTTPBinding again uses a
>>> CachedOutputStream.
>>> This means that the binding cannot be used anymore with longer streams
>>> and
>>> that the streamCache option on the CamelContext bean is ignored. Is this
>>> going to be the way the caching is done in 2.3?
>>>
>>> Thanks,
>>> Jens
>>
>>
>
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for
Camel-2686 and CAMEL-2694 (rev 941275)?
Posted by _Jens <je...@icw.de>.
Hi Willem,
thanks for your quick reply. The issue is that I cannot disable
streamCaching in 2.3-SNAPSHOT anymore. This is what was changed in
DefaultHttpBinding.parseBody:
Before:
// otherwise use input stream
return HttpConverter.toInputStream(request);
Now:
// otherwise use input stream and we need to cache it first
InputStream is = HttpConverter.toInputStream(request,
httpMessage.getExchange());
if (is == null) {
return is;
}
// convert the input stream to StreamCache
try {
CachedOutputStream cos = new
CachedOutputStream(httpMessage.getExchange());
IOHelper.copy(is, cos);
return cos.getStreamCache();
} finally {
is.close();
}
This means, the CachedOutputStream is always used. I have tried to disable
this on the camelContext. But obviously the above code does not check this
option in the camelContext. So it is not possible anymore.
willem.jiang wrote:
>
> These changes just to make sure the we can still read the post message
> body even the body is form message.
>
> You can still configure the CachedOutputStream by using the camelContext
> properties, I don't know what kind of issue that the changes introduced
> to you.
>
> Willem
>
> _Jens wrote:
>> Hi,
>>
>> it seems to me that the DefaultHTTPBinding again uses a
>> CachedOutputStream.
>> This means that the binding cannot be used anymore with longer streams
>> and
>> that the streamCache option on the CamelContext bean is ignored. Is this
>> going to be the way the caching is done in 2.3?
>>
>> Thanks,
>> Jens
>
>
>
--
View this message in context: http://old.nabble.com/2.3-SNAPSHOT%3A-Stream-Caching-again-enabled-by-Bug-Fix-for-Camel-2686-and-CAMEL-2694-%28rev-941275%29--tp28582952p28583243.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: 2.3-SNAPSHOT: Stream Caching again enabled by Bug Fix for Camel-2686
and CAMEL-2694 (rev 941275)?
Posted by Willem Jiang <wi...@gmail.com>.
These changes just to make sure the we can still read the post message
body even the body is form message.
You can still configure the CachedOutputStream by using the camelContext
properties, I don't know what kind of issue that the changes introduced
to you.
Willem
_Jens wrote:
> Hi,
>
> it seems to me that the DefaultHTTPBinding again uses a CachedOutputStream.
> This means that the binding cannot be used anymore with longer streams and
> that the streamCache option on the CamelContext bean is ignored. Is this
> going to be the way the caching is done in 2.3?
>
> Thanks,
> Jens