RE : RE : Multi-threading

["Lefrancois, Carl" <Ca...@axa-canada.com>]

Hello Patrick,

> But if I signal the main thread from the callback, to free the 
> resources, the callback might be switched out directly after this
> signal, and the main thread might free the resources before the
> callback ended and the axis framework used them.

Yes, this is a pretty serious limitation.  I have been trying to find
out if Axis2/C provides a way to either know the thread id of the
communication threads created internally or to be notified exactly when
the threads exit.  As you suggest, this is treated as an implementation
detail and we are expected to code without a precise notification of
when these threads finishes their tasks.  

It sounds like you would like to free resources on a per-call basis.  Is
that right?  I'm quite new to the Axis2/C architecture so perhaps a more
experienced person could suggest a mechanism that would fit into the
existing methodology and provide this extra feature.  My impression is
that the design philosophy so far is to free resources after all calls
are completed.


> With the fd_set in winsock and the select() function, you can wait
> at a maximum of 64 (current implementation) sockets at once.
> With I/O Completion Ports you can use one thread for an infinite
> number of ports... But I think they don't fit well in the modular
> (transportation) design of axis

That's very interesting.  I'm curious as to more of the details of how
this functions... If you have one thread waiting on 12 sockets and want
to make a new call, can this thread begin the next call, or does a
second thread open the socket and pass the job of waiting on it to the
first thread? 

I think we would all agree that your use case would benefit from adding
this capability to Axis2/C.  You mention a potential conflict with the
modular design of Axis; there is also the idea that making such a
powerful feature accessible to the average programmer using Axis could
be a challenge.  Maybe the solution would be to add a new communication
mode instead of changing all asynchronous communication to
one-thread-multi-socket.  I wish I understood the Axis2/C architecture
more fully because this would be an interesting area to contribute.  

The pleasure is all mine in this conversation.  So far I am learning
more about winsock :)
  _____  

"Ce message est confidentiel, a l'usage exclusif du destinataire
ci-dessus et son contenu ne represente en aucun cas un engagement de la
part de AXA, sauf en cas de stipulation expresse et par ecrit de la part
de AXA. Toute publication, utilisation ou diffusion, meme partielle,
doit etre autorisee prealablement. Si vous n'etes pas destinataire de ce
message, merci d'en avertir immediatement l'expediteur."

"This e-mail message is confidential, for the exclusive use of the
addressee and its contents shall not constitute a commitment by AXA,
except as otherwise specifically provided in writing by AXA. Any
unauthorized disclosure, use or dissemination, either whole or partial,
is prohibited. If you are not the intended recipient of the message,
please notify the sender immediately."

Re: RE : RE : Multi-threading

[Damitha Kumarage <da...@gmail.com>]

Patrick van Beem wrote:
> Hello Carl,
>  
>   
>> It sounds like you would like to free resources on a per-call basis.  Is
>> that right?  I'm quite new to the Axis2/C architecture so perhaps a more
>> experienced person could suggest a mechanism that would fit into the
>> existing methodology and provide this extra feature.  My impression is
>> that the design philosophy so far is to free resources after all calls
>> are completed.
>>     
>
> I'm only in it myself for about a month too. But you're right on the design philosophy. Only: that's not the usage pattern of our application. We've got a (very) large client - server application where the user can write his own code (using a custom declarative / constraint programming language). Some interfaces available in this server programming environment perform calls to the outside world using soap. It's up to the user when and how to use it. So our framework must be flexible. We do not know in advance what the end-user is going to write.
> I was hoping to be able to re-use axis structures for each (parallel) call. Or cloning them. But indeed, this is not the design philosophy of axis. I'm now on the road of using thread local storage (TLS) to store the thread-specific structures, so I don't have to allocate / free them for each thread (my threads are worker threads that can do anything. Not just soap calls. So I can't (don't want) to design them for a set of specific soap calls).
>  
>   
>>> With the fd_set in winsock and the select() function, you can wait
>>> at a maximum of 64 (current implementation) sockets at once.
>>> With I/O Completion Ports you can use one thread for an infinite
>>> number of ports... But I think they don't fit well in the modular
>>> (transportation) design of axis
>>>       
>> That's very interesting.  I'm curious as to more of the details of how
>> this functions... If you have one thread waiting on 12 sockets and want
>> to make a new call, can this thread begin the next call, or does a
>> second thread open the socket and pass the job of waiting on it to the
>> first thread? 
>>     
>
> The 'problem' is that the waiting thread can only wait on sockets. Not also on, for example, a job queue. So the waiting thread is mostly implemented as some round-robin algorithm: Wait on the sockets with a time-out of a few milliseconds, check a job queue and optionally perform a few tasks (open more sockets, accept a socket another thread opened, ...) and start waiting again on the new set of sockets. So this uses slightly more CPU then strictly necessary. The IO completion ports use a call-back strategy when IO on a socket is completed. This is a much nicer concept. If you're programming C++, you might be interested in the asio classes of the boost project, where the used IO completion for their windows implementation of the asio (asynchronous IO). But we're running out of context :-)
>
>   
>> I think we would all agree that your use case would benefit from adding
>> this capability to Axis2/C.  You mention a potential conflict with the
>> modular design of Axis; there is also the idea that making such a
>> powerful feature accessible to the average programmer using Axis could
>> be a challenge.  Maybe the solution would be to add a new communication
>> mode instead of changing all asynchronous communication to
>> one-thread-multi-socket.  I wish I understood the Axis2/C architecture
>> more fully because this would be an interesting area to contribute.  
>>     
>
> I'm not 'deep' into the details of axis too (the idea was to use an existing toolkit to save on development time for soap, so I'm not planning to go into much detail for now either). But one implementation might be to add another 'configuration structure' (like the allocator and thread pool) for socket IO and make that responsible for all IO. That implementation can then decide to use one or multiple threads for IO. It can use call-backs to signal the completion (or failure or timeout) of the IO. The async calls can then be implemented as writing data (by the new io struct) and exiting that start-call. Finished. Nothing more to do. No extra thread, nothing. Then, when finished, the call-back can be used to parse the result and call the user call-back for the result. The io struct (module) should probably use a (real!) thread pool for this to prevent one time-consuming call to block other calls. But a simple implementation might to for the 'average' user. This pattern mimics the io completion port / boost interface, so users of axis can easily use these for their async IO.
>   
May be implementing a new transport using Proactor pattern(1) possibly 
using boost library is a good solution. This is possible since new 
transports could be plugged into Axis2/C by design. However this will 
need some changes in op_client implementation because currently for 
async calls it execute on new threads.
However adding this functionality through environment structure seems 
inappropriate. Also it is not clear to me how to implement it in that 
way. WDYT?.

thanks,
Damitha

[1] 
http://www.boost.org/doc/libs/1_37_0/doc/html/boost_asio/overview/core/async.html

-- 
__________________________________________________________________

Damitha Kumarage
http://people.apache.org/
__________________________________________________________________

Re: RE : RE : Multi-threading

[Patrick van Beem <pa...@quintiq.com>]

Hello Carl,
 
> It sounds like you would like to free resources on a per-call basis.  Is
> that right?  I'm quite new to the Axis2/C architecture so perhaps a more
> experienced person could suggest a mechanism that would fit into the
> existing methodology and provide this extra feature.  My impression is
> that the design philosophy so far is to free resources after all calls
> are completed.

I'm only in it myself for about a month too. But you're right on the design philosophy. Only: that's not the usage pattern of our application. We've got a (very) large client - server application where the user can write his own code (using a custom declarative / constraint programming language). Some interfaces available in this server programming environment perform calls to the outside world using soap. It's up to the user when and how to use it. So our framework must be flexible. We do not know in advance what the end-user is going to write.
I was hoping to be able to re-use axis structures for each (parallel) call. Or cloning them. But indeed, this is not the design philosophy of axis. I'm now on the road of using thread local storage (TLS) to store the thread-specific structures, so I don't have to allocate / free them for each thread (my threads are worker threads that can do anything. Not just soap calls. So I can't (don't want) to design them for a set of specific soap calls).
 
>> With the fd_set in winsock and the select() function, you can wait
>> at a maximum of 64 (current implementation) sockets at once.
>> With I/O Completion Ports you can use one thread for an infinite
>> number of ports... But I think they don't fit well in the modular
>> (transportation) design of axis
> 
> That's very interesting.  I'm curious as to more of the details of how
> this functions... If you have one thread waiting on 12 sockets and want
> to make a new call, can this thread begin the next call, or does a
> second thread open the socket and pass the job of waiting on it to the
> first thread? 

The 'problem' is that the waiting thread can only wait on sockets. Not also on, for example, a job queue. So the waiting thread is mostly implemented as some round-robin algorithm: Wait on the sockets with a time-out of a few milliseconds, check a job queue and optionally perform a few tasks (open more sockets, accept a socket another thread opened, ...) and start waiting again on the new set of sockets. So this uses slightly more CPU then strictly necessary. The IO completion ports use a call-back strategy when IO on a socket is completed. This is a much nicer concept. If you're programming C++, you might be interested in the asio classes of the boost project, where the used IO completion for their windows implementation of the asio (asynchronous IO). But we're running out of context :-)

> I think we would all agree that your use case would benefit from adding
> this capability to Axis2/C.  You mention a potential conflict with the
> modular design of Axis; there is also the idea that making such a
> powerful feature accessible to the average programmer using Axis could
> be a challenge.  Maybe the solution would be to add a new communication
> mode instead of changing all asynchronous communication to
> one-thread-multi-socket.  I wish I understood the Axis2/C architecture
> more fully because this would be an interesting area to contribute.  

I'm not 'deep' into the details of axis too (the idea was to use an existing toolkit to save on development time for soap, so I'm not planning to go into much detail for now either). But one implementation might be to add another 'configuration structure' (like the allocator and thread pool) for socket IO and make that responsible for all IO. That implementation can then decide to use one or multiple threads for IO. It can use call-backs to signal the completion (or failure or timeout) of the IO. The async calls can then be implemented as writing data (by the new io struct) and exiting that start-call. Finished. Nothing more to do. No extra thread, nothing. Then, when finished, the call-back can be used to parse the result and call the user call-back for the result. The io struct (module) should probably use a (real!) thread pool for this to prevent one time-consuming call to block other calls. But a simple implementation might to for the 'average' user. This pattern mimics the io completion port / boost interface, so users of axis can easily use these for their async IO.

This whole discussion reminds me that in my opinion, the thread pool in axis should be changed to an API that is accepting jobs that must be performed, instead of providing threads. Threads might not be available at that moment, which now would block the calling thread, which is usually exactly what one does NOT want when using multi threading for asynchronous purposes... The current pool implementation just creates new threads on the flow, and thus does not have this problem. But that's not scalable...

Willing to help workout these concepts...

Regards,

-- 

 
Patrick van Beem
Sr. Software engineer
 
Quintiq
 
T +31 (0) 73 691 07 39
F +31 (0) 73 691 07 54
M +31 (0) 06 15 01 65 83
E patrick.van.beem@quintiq.com
I www.quintiq.com



This message contains information that may be privileged or confidential and is the property of Quintiq. It is only intended for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute or use this message or any part thereof. If you have received this message in error, please notify the sender immediately and delete all copies of this message. Please note that e-mails are susceptible to change, therefore they are not binding.

Re: RE : RE : Multi-threading

[Uthaiyashankar <sh...@wso2.com>]

Hi,


Hello Patrick,
>
> > But if I signal the main thread from the callback, to free the
> > resources, the callback might be switched out directly after this
> > signal, and the main thread might free the resources before the
> > callback ended and the axis framework used them.
>
> Yes, this is a pretty serious limitation.  I have been trying to find
> out if Axis2/C provides a way to either know the thread id of the
> communication threads created internally or to be notified exactly when
> the threads exit.  As you suggest, this is treated as an implementation
> detail and we are expected to code without a precise notification of
> when these threads finishes their tasks.
>
> It sounds like you would like to free resources on a per-call basis.  Is
> that right?  I'm quite new to the Axis2/C architecture so perhaps a more
> experienced person could suggest a mechanism that would fit into the
> existing methodology and provide this extra feature.  My impression is
> that the design philosophy so far is to free resources after all calls
> are completed.
>

Current implementation does not provide a way to free resources on a
per-call basis. I agree with Patrick, we should provide a callback to free
the resources.

Regards,
Shankar


>
>
> > With the fd_set in winsock and the select() function, you can wait
> > at a maximum of 64 (current implementation) sockets at once.
> > With I/O Completion Ports you can use one thread for an infinite
> > number of ports... But I think they don't fit well in the modular
> > (transportation) design of axis
>
> That's very interesting.  I'm curious as to more of the details of how
> this functions... If you have one thread waiting on 12 sockets and want
> to make a new call, can this thread begin the next call, or does a
> second thread open the socket and pass the job of waiting on it to the
> first thread?
>
> I think we would all agree that your use case would benefit from adding
> this capability to Axis2/C.  You mention a potential conflict with the
> modular design of Axis; there is also the idea that making such a
> powerful feature accessible to the average programmer using Axis could
> be a challenge.  Maybe the solution would be to add a new communication
> mode instead of changing all asynchronous communication to
> one-thread-multi-socket.  I wish I understood the Axis2/C architecture
> more fully because this would be an interesting area to contribute.
>
> The pleasure is all mine in this conversation.  So far I am learning
> more about winsock :)
>  _____
>
> "Ce message est confidentiel, a l'usage exclusif du destinataire
> ci-dessus et son contenu ne represente en aucun cas un engagement de la
> part de AXA, sauf en cas de stipulation expresse et par ecrit de la part
> de AXA. Toute publication, utilisation ou diffusion, meme partielle,
> doit etre autorisee prealablement. Si vous n'etes pas destinataire de ce
> message, merci d'en avertir immediatement l'expediteur."
>
> "This e-mail message is confidential, for the exclusive use of the
> addressee and its contents shall not constitute a commitment by AXA,
> except as otherwise specifically provided in writing by AXA. Any
> unauthorized disclosure, use or dissemination, either whole or partial,
> is prohibited. If you are not the intended recipient of the message,
> please notify the sender immediately."
>



-- 
S.Uthaiyashankar
Software Architect
WSO2 Inc.
http://wso2.com/ - "The Open Source SOA Company"