Replication thread handling : a proposal

[Emmanuel Lécharny <el...@gmail.com>]

Hi,

as a follow up to my previous mail, here is what I propose to improve
the consumer handling :

- the server starts as many thread as we have consumers
- those threads will loop forever, until we stop the server, or until we
explicitely shutdown the consumer (an extended request could be used for
that, but I'd rather see the use of a dynamic configuration, in a near
future : stopping or startin the consumer will just be a matter of
changing an AT in the config partition)
- each consumer will be initialized outside of this thread, but started
in the thread

Now, the thread itself. We have to kind of replication :
A) RefreshOnly
1- loop until connected (wait 5 seconds if not)
2- call doSyncSearch
3- if this method returns, we restart at (1) if disconnected, or at (2),
with a wait delay of RefreshInterval

B) RefreshAndPersist
1- loop until connected (wait 5 seconds if not)
2- call doSyncSearch
3- if this method returns, we restart at (1) if disconnected, or at (2),
with a wait delay that depends on the return code :
 - RefreshInterval if we've got an error
 - no delay if a full refresh is required


Here, we don't start a new thread, we use the thread we are running in.
If we get disconnected at some point, the MINA connectionClosed() method
will be called, and we will exit from the doSyncSearch method, and call
back the connect method.

As we can see, it simplifies the code, as we only use one single thread
per consumer, and most of the code is shared, if we except the delay we
have to wait (of course, the doSyncSearch will behave accordingly to the
replication mode).

thoughts ?

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com 

Re: Replication thread handling : a proposal

[Kiran Ayyagari <ka...@apache.org>]

On Mon, Dec 3, 2012 at 7:47 PM, Emmanuel Lécharny <el...@gmail.com>wrote:

> Hi,
>
> as a follow up to my previous mail, here is what I propose to improve
> the consumer handling :
>
> - the server starts as many thread as we have consumers
> - those threads will loop forever, until we stop the server, or until we
> explicitely shutdown the consumer (an extended request could be used for
> that, but I'd rather see the use of a dynamic configuration, in a near
> future : stopping or startin the consumer will just be a matter of
> changing an AT in the config partition)
> - each consumer will be initialized outside of this thread, but started
> in the thread
>
> Now, the thread itself. We have to kind of replication :
> A) RefreshOnly
> 1- loop until connected (wait 5 seconds if not)
> 2- call doSyncSearch
> 3- if this method returns, we restart at (1) if disconnected, or at (2),
> with a wait delay of RefreshInterval
>
> B) RefreshAndPersist
> 1- loop until connected (wait 5 seconds if not)
> 2- call doSyncSearch
> 3- if this method returns, we restart at (1) if disconnected, or at (2),
> with a wait delay that depends on the return code :
>  - RefreshInterval if we've got an error
>  - no delay if a full refresh is required
>
>
> Here, we don't start a new thread, we use the thread we are running in.
> If we get disconnected at some point, the MINA connectionClosed() method
> will be called, and we will exit from the doSyncSearch method, and call
> back the connect method.
>
> As we can see, it simplifies the code, as we only use one single thread
> per consumer, and most of the code is shared, if we except the delay we
> have to wait (of course, the doSyncSearch will behave accordingly to the
> replication mode).
>
> thoughts ?
>
> sounds good to me.

P.S:- the additional thread that gets created in refreshOnly mode was added
to counter some blocking issue
        I had encountered while implementing the initial version, not sure
if that issue still exists now.
       (Just a hint to explain the reason for this additional thread's
existence)

> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>


-- 
Kiran Ayyagari
http://keydap.com