High availability of local listeners for ContinuousQuery or Events

[38797715 <38...@qq.com>]

Hi community,

For local listeners registered for ContinuousQuery and Events, is there 
a corresponding high availability mechanism design? That is, if the node 
registering the local listener fails, what state will the cluster be?

If do not register a local listener, but write the business code in the 
remote filter and return false, is this a good practice?

Re: High availability of local listeners for ContinuousQuery or Events

[Stanislav Lukyanov <st...@gmail.com>]

Hi,

It's true that currently you need to implement something for Continuous Queries failover in your application code.
Continuous Queries have setInitialQuery API to help with that.

How it's supposed to work is: you define an initial query (e.g. an SQL query) which fetches the data that's already in the cache when the CQ is being registered.
In case of failover, your initial query should return the data your CQ listener has missed (or might have missed) while it was down.
For example, the following design would work
- The cache has a "last update time" field and an SQL index over that field
- The initial query of the CQ queries all data updated in the last 30 minutes; it will be fast because of the index
- After you've started the CQ, you process the initial query results; if your CQ listener was down for less than 30 minutes, it will receive all updates it has missed (plus some that it has already seen - it needs to filter them out or just be idempotent)

I'm sure that an out-of-the-box HA mechanism for CQ listeners will be added in the future versions of the platform.

Stan

> On 30 Oct 2020, at 11:01, 38797715 <38...@qq.com> wrote:
> 
> Hi Igor,
> 
> We hope that if the local listener node fails, we can have a mechanism similar to fail over. Otherwise, if the local listener node fails and restarts, the events during the failure will be lost.
> 
> 在 2020/10/30 上午12:04, Igor Belyakov 写道:
>> Hi,
>> 
>> In case the node, which registered a continuous query fails, the continuous query will be undeployed from the cluster. The cluster state won't be changed.
>> 
>> It's not a good practice to write the business code in a remote filter. Could you please clarify more details regarding your use case?
>> 
>> Igor
>> 
>> On Thu, Oct 29, 2020 at 4:46 PM 38797715 <38797715@qq.com <ma...@qq.com>> wrote:
>> Hi community,
>> 
>> For local listeners registered for ContinuousQuery and Events, is there 
>> a corresponding high availability mechanism design? That is, if the node 
>> registering the local listener fails, what state will the cluster be?
>> 
>> If do not register a local listener, but write the business code in the 
>> remote filter and return false, is this a good practice?
>> 
>> 

Re: High availability of local listeners for ContinuousQuery or Events

[38797715 <38...@qq.com>]

Hi Igor,

We hope that if the local listener node fails, we can have a mechanism 
similar to fail over. Otherwise, if the local listener node fails and 
restarts, the events during the failure will be lost.

在 2020/10/30 上午12:04, Igor Belyakov 写道:
> Hi,
>
> In case the node, which registered a continuous query fails, the 
> continuous query will be undeployed from the cluster. The cluster 
> state won't be changed.
>
> It's not a good practice to write the business code in a remote 
> filter. Could you please clarify more details regarding your use case?
>
> Igor
>
> On Thu, Oct 29, 2020 at 4:46 PM 38797715 <38797715@qq.com 
> <ma...@qq.com>> wrote:
>
>     Hi community,
>
>     For local listeners registered for ContinuousQuery and Events, is
>     there
>     a corresponding high availability mechanism design? That is, if
>     the node
>     registering the local listener fails, what state will the cluster be?
>
>     If do not register a local listener, but write the business code
>     in the
>     remote filter and return false, is this a good practice?
>
>

Re: High availability of local listeners for ContinuousQuery or Events

[Igor Belyakov <ig...@gmail.com>]

Hi,

In case the node, which registered a continuous query fails, the continuous
query will be undeployed from the cluster. The cluster state won't be
changed.

It's not a good practice to write the business code in a remote filter.
Could you please clarify more details regarding your use case?

Igor

On Thu, Oct 29, 2020 at 4:46 PM 38797715 <38...@qq.com> wrote:

> Hi community,
>
> For local listeners registered for ContinuousQuery and Events, is there
> a corresponding high availability mechanism design? That is, if the node
> registering the local listener fails, what state will the cluster be?
>
> If do not register a local listener, but write the business code in the
> remote filter and return false, is this a good practice?
>
>
>