You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@synapse.apache.org by Hiranya Jayathilaka <hi...@gmail.com> on 2013/07/18 22:07:16 UTC
Bug in VFS Transport - SYNAPSE-851
Hi Folks,
I'm in the process of fixing SYNAPSE-851. There is a problem in the VFS transport if you try to mediate a non-XML message through Synapse without touching its payload (i.e. pass through). When Synapse tries to send the message out, Axis2 will spawn a new thread to handle the send operation. At this point, the VFS listener thread that received the message gets released and it closes the file input stream. Since the message hasn't been built yet, Synapse will end up writing an empty message to the output stream.
There are 2 workarounds to the problem:
1. Force Synapse to build the message in the in-sequence by engaging some mediator (e.g. log mediator).
2. Remove the ClientAPINonBlocking property (successor to the old transportNonBlocking property) from the message which forces Axis2 to send the message out on the same VFS listener thread.
My question is, should we try to do better here? This problem must have always been there, but usually we do some processing on the incoming message (e.g. XSLT), which circumvents the issue. So we can treat pass through scenario as a special case and make it a requirement to remove the ClientAPINonBlockingProperty. Or we can make it the default behavior of the VFS transport listener. We can even try to build all VFS requests before they are sent out to any endpoint.
WDYT?
Thanks,
Hiranya
--
Hiranya Jayathilaka
Mayhem Lab/RACE Lab;
Dept. of Computer Science, UCSB; http://cs.ucsb.edu
E-mail: hiranya@cs.ucsb.edu; Mobile: +1 (805) 895-7443
Blog: http://techfeast-hiranya.blogspot.com
Re: Bug in VFS Transport - SYNAPSE-851
Posted by Amila Maha Arachchi <am...@wso2.com>.
On Fri, Jul 19, 2013 at 7:54 PM, Charith Wickramarachchi <
charith.dhanushka@gmail.com> wrote:
> I think we must keep the streaming behavior of VFS transport. I have seen
> use cases where people use VFS transport to manage legacy content
> delivery/routing which involves large file transfers.
>
Yes, this will become a problem for large file processing scenarios. Hence,
we shouldn't force to build the message.
>
> Do you see any down side of forcing to remove ClientAPINonBlockingProperty
> at the transport level in the VFS transport. I think its acceptable for
> most of the use cases.
>
> regards,
> Charith
>
>
>
> On Thu, Jul 18, 2013 at 4:07 PM, Hiranya Jayathilaka <hiranya911@gmail.com
> > wrote:
>
>> Hi Folks,
>>
>> I'm in the process of fixing SYNAPSE-851. There is a problem in the VFS
>> transport if you try to mediate a non-XML message through Synapse without
>> touching its payload (i.e. pass through). When Synapse tries to send the
>> message out, Axis2 will spawn a new thread to handle the send operation. At
>> this point, the VFS listener thread that received the message gets released
>> and it closes the file input stream. Since the message hasn't been built
>> yet, Synapse will end up writing an empty message to the output stream.
>>
>> There are 2 workarounds to the problem:
>> 1. Force Synapse to build the message in the in-sequence by engaging some
>> mediator (e.g. log mediator).
>> 2. Remove the ClientAPINonBlocking property (successor to the old
>> transportNonBlocking property) from the message which forces Axis2 to send
>> the message out on the same VFS listener thread.
>>
>> My question is, should we try to do better here? This problem must have
>> always been there, but usually we do some processing on the incoming
>> message (e.g. XSLT), which circumvents the issue. So we can treat pass
>> through scenario as a special case and make it a requirement to remove the
>> ClientAPINonBlockingProperty. Or we can make it the default behavior of the
>> VFS transport listener. We can even try to build all VFS requests before
>> they are sent out to any endpoint.
>>
>> WDYT?
>>
>> Thanks,
>> Hiranya
>> --
>> Hiranya Jayathilaka
>> Mayhem Lab/RACE Lab;
>> Dept. of Computer Science, UCSB; http://cs.ucsb.edu
>> E-mail: hiranya@cs.ucsb.edu <hi...@wso2.com>; Mobile: +1 (805)
>> 895-7443
>> Blog: http://techfeast-hiranya.**blogspot.com<http://techfeast-hiranya.blogspot.com/>
>>
>>
>
>
> --
> Charith Dhanushka Wickramaarachchi
>
> Tel +1 213 447 4253
> Web http://apache.org/~charith <http://www-scf.usc.edu/~cwickram/><http://charith.wickramaarachchi.org/>
> Blog http://charith.wickramaarachchi.org/<http://charithwiki.blogspot.com/>
> Twitter @charithwiki <https://twitter.com/charithwiki>
>
--
*Amila Maharachchi*
Senior Technical Lead
WSO2, Inc.; http://wso2.com
Blog: http://maharachchi.blogspot.com
Mobile: +94719371446
Re: Bug in VFS Transport - SYNAPSE-851
Posted by Hiranya Jayathilaka <hi...@gmail.com>.
On Jul 19, 2013, at 7:24 AM, Charith Wickramarachchi <ch...@gmail.com> wrote:
> I think we must keep the streaming behavior of VFS transport. I have seen use cases where people use VFS transport to manage legacy content delivery/routing which involves large file transfers.
+1
>
> Do you see any down side of forcing to remove ClientAPINonBlockingProperty at the transport level in the VFS transport. I think its acceptable for most of the use cases.
I can't seem to find a scenario where this would cause any adverse effects. So if there are no objections, I'd go ahead and make that change.
Also I think this issue is related to SYNAPSE-527. There Andreas has suggested we do not copy this property to the outgoing message, which will also take care of this issue in the VFS transport. The problem with that is, if the user wants to modify this property value from the Synapse configuration, those changes will probably get discarded.
Thanks,
Hiranya
>
> regards,
> Charith
>
>
>
> On Thu, Jul 18, 2013 at 4:07 PM, Hiranya Jayathilaka <hi...@gmail.com> wrote:
> Hi Folks,
>
> I'm in the process of fixing SYNAPSE-851. There is a problem in the VFS transport if you try to mediate a non-XML message through Synapse without touching its payload (i.e. pass through). When Synapse tries to send the message out, Axis2 will spawn a new thread to handle the send operation. At this point, the VFS listener thread that received the message gets released and it closes the file input stream. Since the message hasn't been built yet, Synapse will end up writing an empty message to the output stream.
>
> There are 2 workarounds to the problem:
> 1. Force Synapse to build the message in the in-sequence by engaging some mediator (e.g. log mediator).
> 2. Remove the ClientAPINonBlocking property (successor to the old transportNonBlocking property) from the message which forces Axis2 to send the message out on the same VFS listener thread.
>
> My question is, should we try to do better here? This problem must have always been there, but usually we do some processing on the incoming message (e.g. XSLT), which circumvents the issue. So we can treat pass through scenario as a special case and make it a requirement to remove the ClientAPINonBlockingProperty. Or we can make it the default behavior of the VFS transport listener. We can even try to build all VFS requests before they are sent out to any endpoint.
>
> WDYT?
>
> Thanks,
> Hiranya
> --
> Hiranya Jayathilaka
> Mayhem Lab/RACE Lab;
> Dept. of Computer Science, UCSB; http://cs.ucsb.edu
> E-mail: hiranya@cs.ucsb.edu; Mobile: +1 (805) 895-7443
> Blog: http://techfeast-hiranya.blogspot.com
>
>
>
>
> --
> Charith Dhanushka Wickramaarachchi
>
> Tel +1 213 447 4253
> Web http://apache.org/~charith
> Blog http://charith.wickramaarachchi.org/
> Twitter @charithwiki
--
Hiranya Jayathilaka
Mayhem Lab/RACE Lab;
Dept. of Computer Science, UCSB; http://cs.ucsb.edu
E-mail: hiranya@cs.ucsb.edu; Mobile: +1 (805) 895-7443
Blog: http://techfeast-hiranya.blogspot.com
Re: Bug in VFS Transport - SYNAPSE-851
Posted by Charith Wickramarachchi <ch...@gmail.com>.
I think we must keep the streaming behavior of VFS transport. I have seen
use cases where people use VFS transport to manage legacy content
delivery/routing which involves large file transfers.
Do you see any down side of forcing to remove ClientAPINonBlockingProperty
at the transport level in the VFS transport. I think its acceptable for
most of the use cases.
regards,
Charith
On Thu, Jul 18, 2013 at 4:07 PM, Hiranya Jayathilaka
<hi...@gmail.com>wrote:
> Hi Folks,
>
> I'm in the process of fixing SYNAPSE-851. There is a problem in the VFS
> transport if you try to mediate a non-XML message through Synapse without
> touching its payload (i.e. pass through). When Synapse tries to send the
> message out, Axis2 will spawn a new thread to handle the send operation. At
> this point, the VFS listener thread that received the message gets released
> and it closes the file input stream. Since the message hasn't been built
> yet, Synapse will end up writing an empty message to the output stream.
>
> There are 2 workarounds to the problem:
> 1. Force Synapse to build the message in the in-sequence by engaging some
> mediator (e.g. log mediator).
> 2. Remove the ClientAPINonBlocking property (successor to the old
> transportNonBlocking property) from the message which forces Axis2 to send
> the message out on the same VFS listener thread.
>
> My question is, should we try to do better here? This problem must have
> always been there, but usually we do some processing on the incoming
> message (e.g. XSLT), which circumvents the issue. So we can treat pass
> through scenario as a special case and make it a requirement to remove the
> ClientAPINonBlockingProperty. Or we can make it the default behavior of the
> VFS transport listener. We can even try to build all VFS requests before
> they are sent out to any endpoint.
>
> WDYT?
>
> Thanks,
> Hiranya
> --
> Hiranya Jayathilaka
> Mayhem Lab/RACE Lab;
> Dept. of Computer Science, UCSB; http://cs.ucsb.edu
> E-mail: hiranya@cs.ucsb.edu <hi...@wso2.com>; Mobile: +1 (805) 895-7443
> Blog: http://techfeast-hiranya.**blogspot.com<http://techfeast-hiranya.blogspot.com/>
>
>
--
Charith Dhanushka Wickramaarachchi
Tel +1 213 447 4253
Web http://apache.org/~charith
<http://www-scf.usc.edu/~cwickram/><http://charith.wickramaarachchi.org/>
Blog http://charith.wickramaarachchi.org/<http://charithwiki.blogspot.com/>
Twitter @charithwiki <https://twitter.com/charithwiki>