You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by java kurious <ja...@gmail.com> on 2013/10/10 23:13:35 UTC
Netty: messages not propagated back from netty producer
Hi All,
I am using netty to send a msg to a remote socket, and process the
response.
Consider the route below:
1. <camel:from uri="direct:msg_in_queue"/>
2. <log message="input message received"/>
3. <to
uri="xslt:xslts/msg-transform.xsl"/>
4. <!-- Send to remote socket -->
5. <to uri="netty:tcp://
192.168.1.1:5511?textline=true&sync=true&keepAlive=false"/>
6. <to uri="log:response from netty
?showHeaders=false"/>
7. <to uri="xslt:xslts/msg-transform-back.xsl"/>
I receive a webservice request in XML format, and I pass it to *msg_in_queue
* endpoint. From, there it gets transformed and sent to *netty* component.
The response from netty is logged correctly in step 6. and it gets
transformed in #7, but the message is not getting propagated back to
webservice caller.
If I comment out step 5, then the messages is getting propagated back and
the caller sees the response, though it is not the messages which is
expected because step #5 is commented out.
The point is why netty component is causing this behavior.
I am using camel. 2.9.1
Thanks
Re: Netty: messages not propagated back from netty producer
Posted by James Carman <ja...@carmanconsulting.com>.
I guess there's also the concept of what makes up a response. How
does the endpoint know how many bytes are included in the response
message from the server?
On Sun, Oct 13, 2013 at 8:42 PM, javakurious <ja...@gmail.com> wrote:
> If I comment out the netty component, then everything works fine. So, CXF
> knows how to connect a request to its response.
>
> And I was hoping that Netty component does the same. What I don't understand
> is why setting the synchronous parameter to true is not making Netty wait
> for the response..
>
> Thanks
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741488.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
Resolved !!!!
The problem was the following code in the remote socket server code:
PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
StringBuilder msg = new
StringBuilder(message.getMessage().replaceAll("[\n\r]", ""));
msg.append("\n");
out.println(msg.toString());
Since, I was using textline codec and the remote server was returning "\n"
at the end of the message and using println() to write the message, the
consumer netty was reading the empty body.
Though it still does not completely explain why I would see the correct
message once in a while, and while I see the response in debug mode.
However, changing from println() to print() seems to have solved the issue
for now.
Thanks a lot for your help. I really appreciate it.
Thanks
Vineet
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5742282.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
There was a fix about request timeout recently. See CAMEL-6730
On Fri, Oct 25, 2013 at 10:55 PM, javakurious <ja...@gmail.com> wrote:
> In debug mode, it runs fine. So, it is a timing issue.. Netty producer is not
> waiting for the response even if I set the requestTimeout value to be 10
> secs.
>
> Vineet
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5742227.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
In debug mode, it runs fine. So, it is a timing issue.. Netty producer is not
waiting for the response even if I set the requestTimeout value to be 10
secs.
Vineet
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5742227.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
Another observation:
If I set the synchronous to true, I get empty body back..
Could netty be receiving something back before the actual response, which
makes it just return right away.
Vineet
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5742226.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
Sorry for responding so late..
That didn't help either. noticed that when I sent an erroneous message to
remote netty component, which returned right away with a failure message, it
was received and displayed correctly. But when I send a correct message ,
and the remote server is processing the message, it is not received back..
It really feels like that netty producer is not waiting for the response,
when the remote netty server is processing the request..
Vineet
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5742218.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
I tried that. Didn't change the behavior..
I see the response being logged correctly step #C (<log message="response
body - ${body}"/>).
<cxf:cxfEndpoint id="ndaEndpointsoap12"
address="/soap12/search"
serviceClass="com.personal.Search">
<cxf:binding>
<soap:soapBinding mtomEnabled="true" version="1.2"/>
</cxf:binding>
</cxf:cxfEndpoint>
<route id="soap12">
A. <from uri="cxf:bean:ndaEndpointsoap12" />
B. <to uri="direct:msg_in_queue"/>
C.<log message="response body - ${body}"/>
</route>
<camel:route id="msg-handling-route" streamCache="true">
D. <camel:from uri="direct:msg_in_queue"/>
E. <log message="input message received"/>
F. <to uri="xslt:xslts/msg-transform.xsl"/>
G. <to
uri="netty:tcp://192.168.1.1:5511?textline=true&decoderMaxLineLength=10000&sync=true&keepAlive=true"/>
H.<to uri="log:response from netty?showHeaders=false"/>
I . <to uri="xslt:xslts/msg-transform-back.xsl"/>
J. <convertBodyTo type="String"/>
</camel:route>
Thanks
Vineet
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741546.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by Claus Ibsen <cl...@gmail.com>.
Can you try move the convert body after netty, eg between G and H.
On Mon, Oct 14, 2013 at 4:38 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Can you try adding
>
> <convertBodyTo type="String"/> at the end of the route, to ensure the
> message is loaded into memory and not stream based, so the CXF
> consumer uses that as reply.
>
>
>
> On Mon, Oct 14, 2013 at 4:25 PM, javakurious <ja...@gmail.com> wrote:
>> Thanks for your patience and help..
>>
>> Yes, the response from Netty (when synchronous = false) is always logged in
>> #6 with or without Netty. But the response is not always received back by
>> cxf consumer. The webservice client sees the response, say, 1 out of 10
>> times.
>>
>> However, if I comment out Netty, everything flows perfectly, in terms of
>> every component seeing the body.
>>
>> I enabled streamCaching by setting streamCache="true" in the route, but that
>> did not help much. Though, it does seem that the WebService client might be
>> seeing the response a little more frequently. Before, it probably was
>> something like 1 out of 15 times. But that could be just me imagining
>> things..
>>
>> Thanks.
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741537.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: cibsen@redhat.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Netty: messages not propagated back from netty producer
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Can you try adding
<convertBodyTo type="String"/> at the end of the route, to ensure the
message is loaded into memory and not stream based, so the CXF
consumer uses that as reply.
On Mon, Oct 14, 2013 at 4:25 PM, javakurious <ja...@gmail.com> wrote:
> Thanks for your patience and help..
>
> Yes, the response from Netty (when synchronous = false) is always logged in
> #6 with or without Netty. But the response is not always received back by
> cxf consumer. The webservice client sees the response, say, 1 out of 10
> times.
>
> However, if I comment out Netty, everything flows perfectly, in terms of
> every component seeing the body.
>
> I enabled streamCaching by setting streamCache="true" in the route, but that
> did not help much. Though, it does seem that the WebService client might be
> seeing the response a little more frequently. Before, it probably was
> something like 1 out of 15 times. But that could be just me imagining
> things..
>
> Thanks.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741537.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
Thanks for your patience and help..
Yes, the response from Netty (when synchronous = false) is always logged in
#6 with or without Netty. But the response is not always received back by
cxf consumer. The webservice client sees the response, say, 1 out of 10
times.
However, if I comment out Netty, everything flows perfectly, in terms of
every component seeing the body.
I enabled streamCaching by setting streamCache="true" in the route, but that
did not help much. Though, it does seem that the WebService client might be
seeing the response a little more frequently. Before, it probably was
something like 1 out of 15 times. But that could be just me imagining
things..
Thanks.
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741537.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by Claus Ibsen <cl...@gmail.com>.
Just to back up to the start.
In your fist post, at #6. Do you see this logging always? eg with and
without netty?
As if you see #6 then you have a response, and the CXF consumer should
be able to use that as its response.
Just mind about streams, so you may need to enable stream caching
http://camel.apache.org/why-is-my-message-body-empty.html
http://camel.apache.org/stream-caching.html
Or remove the log, or do not show the body etc.
On Mon, Oct 14, 2013 at 2:42 AM, javakurious <ja...@gmail.com> wrote:
> If I comment out the netty component, then everything works fine. So, CXF
> knows how to connect a request to its response.
>
> And I was hoping that Netty component does the same. What I don't understand
> is why setting the synchronous parameter to true is not making Netty wait
> for the response..
>
> Thanks
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741488.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
If I comment out the netty component, then everything works fine. So, CXF
knows how to connect a request to its response.
And I was hoping that Netty component does the same. What I don't understand
is why setting the synchronous parameter to true is not making Netty wait
for the response..
Thanks
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741488.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Netty: messages not propagated back from netty producer
Posted by James Carman <ja...@carmanconsulting.com>.
How is the component going to correlate the request with the response?
On Sunday, October 13, 2013, javakurious wrote:
> Upgraded the camel to 2.12.1 .
>
> No effect ! :(
>
> Here is the reiteration of the issue and a couple of additional
> observations:
>
> Route:
> I am getting a webservice request using CXF, I send the request to a remote
> server using netty and pass the response back to the webservice client.
>
>
> Given that the default behavior of Netty is async, it's causing a
> disconnect
> between CXF request and the response returned by Netty. Therefore, the
> webservice keeps getting empty response, even though the remote socket is
> responding to Netty , but its getting lost. If the webservice client keep
> sending requests, every once in a while, it actually sees a response.
>
> So, I set the synchronous parameter to be true. Now, Netty component keeps
> returning empty.It appears that Netty is not waiting for the response from
> remote server, and just moves on to next component. Is there a way to make
> netty block until it receives a response from remote socket ?
>
> Thanks
>
>
>
>
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741486.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
Upgraded the camel to 2.12.1 .
No effect ! :(
Here is the reiteration of the issue and a couple of additional
observations:
Route:
I am getting a webservice request using CXF, I send the request to a remote
server using netty and pass the response back to the webservice client.
Given that the default behavior of Netty is async, it's causing a disconnect
between CXF request and the response returned by Netty. Therefore, the
webservice keeps getting empty response, even though the remote socket is
responding to Netty , but its getting lost. If the webservice client keep
sending requests, every once in a while, it actually sees a response.
So, I set the synchronous parameter to be true. Now, Netty component keeps
returning empty.It appears that Netty is not waiting for the response from
remote server, and just moves on to next component. Is there a way to make
netty block until it receives a response from remote socket ?
Thanks
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741486.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by Claus Ibsen <cl...@gmail.com>.
Maybe not. Try upgrading Camel and some of the suggestion we say on this page
http://camel.apache.org/support
On Fri, Oct 11, 2013 at 8:36 PM, javakurious <ja...@gmail.com> wrote:
> Thanks for the clarification..
>
> So, that won't have anything to do with the message not being returned up
> the chain back to the caller. would it ?
>
> Thanks
> Vineet
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741372.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
Thanks for the clarification..
So, that won't have anything to do with the message not being returned up
the chain back to the caller. would it ?
Thanks
Vineet
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741372.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Netty: messages not propagated back from netty producer
Posted by Claus Ibsen <cl...@gmail.com>.
Yeah the sync option is sorta inherited from camel-mina many many years ago.
Its really for doing request/reply.
But the threading model can still be asynchronous etc.
Using synchronous is a general option you can use on any endpoint.
That forces the threading model to be synchronous.
On Fri, Oct 11, 2013 at 6:08 PM, javakurious <ja...@gmail.com> wrote:
> Just a quick clarification for Netty..
>
> What's the difference between sync and synchronous parameter ?
>
> Thanks
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741364.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Netty: messages not propagated back from netty producer
Posted by javakurious <ja...@gmail.com>.
Just a quick clarification for Netty..
What's the difference between sync and synchronous parameter ?
Thanks
--
View this message in context: http://camel.465427.n5.nabble.com/Netty-messages-not-propagated-back-from-netty-producer-tp5741315p5741364.html
Sent from the Camel - Users mailing list archive at Nabble.com.