You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Ro...@t-systems.com on 2020/06/05 03:44:04 UTC
Continuous Queries with several remote filter on the same cache
Hi Community,
I ask this question here because I haven't found the answer in the documentation.
Could you please clarify how Continuous Queries work? What the behavior of Continuous Queries if we have several clients with different Remote Filters on the same cache? For example, if we have: one server node with cache and we have up to 20 client nodes each of them will execute Continuous Query on the same cache but with different Remote Filters. Will each client get the data according to its remote filter? Or it is supposed to have only one Remote Filter for all clients and every client should filter data in its local event listener?
I would be grateful if you send some link which describes the behavior of Continuous Queries more thoroughly.
Best regards,
Roman
Re: Continuous Queries with several remote filter on the same cache
Posted by Denis Magda <dm...@apache.org>.
Hi Roman,
Apologize for the late reply. Could you please clarify why it's useful to
the threads count parameter? How are you planning to use it?
-
Denis
On Tue, Jul 7, 2020 at 5:25 AM <Ro...@t-systems.com> wrote:
> Hi Denis,
>
> What do you think about some improvements in @IgniteAsyncCallback
> regarding usability? What if we add a number of threads parameter in
> @IgniteAsyncCallback?
>
> Best regards,
> Roman
>
> -----Original Message-----
> From: Denis Magda <dm...@apache.org>
> Sent: Friday, June 26, 2020 7:38 PM
> To: dev <de...@ignite.apache.org>
> Subject: Re: Continuous Queries with several remote filter on the same
> cache
>
> Roman,
>
> The updates are ordered per partition. Let's take this example of an
> application updating several records:
>
> put (k1, val1) => mapped to partition_10 => node_A put (k2, val2) =>
> mapped to partition_5 => node_B put (k3, val3) => mapped to partition_10 =>
> node_A
>
> It's guaranteed that a continuous query listener will be notified about k1
> and k3 updates in this order - k1 first and k3 after. As for the k2 update,
> it can arrive at any time (i.e., before k1, after k3 or in the middle).
>
>
>
>
> -
> Denis
>
>
> On Fri, Jun 26, 2020 at 12:58 AM <Ro...@t-systems.com> wrote:
>
> > Hi Denis,
> >
> > Thanks! Is there some guarantee about the order of the updates? Even
> > when we have multiple cache nodes.
> >
> > Best regards,
> > Roman
> >
> > -----Original Message-----
> > From: Denis Magda <dm...@apache.org>
> > Sent: Monday, June 8, 2020 10:20 PM
> > To: dev <de...@ignite.apache.org>
> > Subject: Re: Continuous Queries with several remote filter on the same
> > cache
> >
> > Roman,
> >
> > Please check the following methods:
> > * CacheContiniousQueryHandler (the filter usage):
> >
> > https://github.com/apache/ignite/blob/6955ac291352dd67c1f84a006cda512e
> > e54f38bb/modules/core/src/main/java/org/apache/ignite/internal/process
> > ors/cache/query/continuous/CacheContinuousQueryHandler.java#L994
> > * CacheContinuousQueryManager (the listener execution):
> >
> > https://github.com/apache/ignite/blob/master/modules/core/src/main/jav
> > a/org/apache/ignite/internal/processors/cache/query/continuous/CacheCo
> > ntinuousQueryManager.java#L376
> >
> > -
> > Denis
> >
> >
> > On Sun, Jun 7, 2020 at 12:19 AM <Ro...@t-systems.com> wrote:
> >
> > > Hi Denis,
> > > A big thank you for the answer.
> > > Could you please tell me where can I find this logic in the sources.
> > > Which package should I look into?
> > >
> > > -----Original Message-----
> > > From: Denis Magda <dm...@apache.org>
> > > Sent: Saturday, June 6, 2020 2:07 AM
> > > To: dev <de...@ignite.apache.org>
> > > Subject: Re: Continuous Queries with several remote filter on the
> > > same cache
> > >
> > > Hi Roman,
> > >
> > > Every continuous query is a unique entity that is processed by
> > > servers independently. With your example, the server node will
> > > execute all 20 filters for every cache insert/update operation. The
> > > server will notify through local listeners only those clients whose
> > > remote filters returned 'true'.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Thu, Jun 4, 2020 at 8:44 PM <Ro...@t-systems.com> wrote:
> > >
> > > > Hi Community,
> > > >
> > > > I ask this question here because I haven't found the answer in the
> > > > documentation.
> > > >
> > > > Could you please clarify how Continuous Queries work? What the
> > > > behavior of Continuous Queries if we have several clients with
> > > > different Remote Filters on the same cache? For example, if we have:
> > > > one server node with cache and we have up to 20 client nodes each
> > > > of them will execute Continuous Query on the same cache but with
> > > > different Remote Filters. Will each client get the data according
> > > > to its remote filter? Or it is supposed to have only one Remote
> > > > Filter for all clients and every client should filter data in its
> > > > local event
> > > listener?
> > > > I would be grateful if you send some link which describes the
> > > > behavior of Continuous Queries more thoroughly.
> > > > Best regards,
> > > > Roman
> > > >
> > >
> >
>
RE: Continuous Queries with several remote filter on the same cache
Posted by Ro...@t-systems.com.
Hi Denis,
What do you think about some improvements in @IgniteAsyncCallback regarding usability? What if we add a number of threads parameter in @IgniteAsyncCallback?
Best regards,
Roman
-----Original Message-----
From: Denis Magda <dm...@apache.org>
Sent: Friday, June 26, 2020 7:38 PM
To: dev <de...@ignite.apache.org>
Subject: Re: Continuous Queries with several remote filter on the same cache
Roman,
The updates are ordered per partition. Let's take this example of an application updating several records:
put (k1, val1) => mapped to partition_10 => node_A put (k2, val2) => mapped to partition_5 => node_B put (k3, val3) => mapped to partition_10 => node_A
It's guaranteed that a continuous query listener will be notified about k1 and k3 updates in this order - k1 first and k3 after. As for the k2 update, it can arrive at any time (i.e., before k1, after k3 or in the middle).
-
Denis
On Fri, Jun 26, 2020 at 12:58 AM <Ro...@t-systems.com> wrote:
> Hi Denis,
>
> Thanks! Is there some guarantee about the order of the updates? Even
> when we have multiple cache nodes.
>
> Best regards,
> Roman
>
> -----Original Message-----
> From: Denis Magda <dm...@apache.org>
> Sent: Monday, June 8, 2020 10:20 PM
> To: dev <de...@ignite.apache.org>
> Subject: Re: Continuous Queries with several remote filter on the same
> cache
>
> Roman,
>
> Please check the following methods:
> * CacheContiniousQueryHandler (the filter usage):
>
> https://github.com/apache/ignite/blob/6955ac291352dd67c1f84a006cda512e
> e54f38bb/modules/core/src/main/java/org/apache/ignite/internal/process
> ors/cache/query/continuous/CacheContinuousQueryHandler.java#L994
> * CacheContinuousQueryManager (the listener execution):
>
> https://github.com/apache/ignite/blob/master/modules/core/src/main/jav
> a/org/apache/ignite/internal/processors/cache/query/continuous/CacheCo
> ntinuousQueryManager.java#L376
>
> -
> Denis
>
>
> On Sun, Jun 7, 2020 at 12:19 AM <Ro...@t-systems.com> wrote:
>
> > Hi Denis,
> > A big thank you for the answer.
> > Could you please tell me where can I find this logic in the sources.
> > Which package should I look into?
> >
> > -----Original Message-----
> > From: Denis Magda <dm...@apache.org>
> > Sent: Saturday, June 6, 2020 2:07 AM
> > To: dev <de...@ignite.apache.org>
> > Subject: Re: Continuous Queries with several remote filter on the
> > same cache
> >
> > Hi Roman,
> >
> > Every continuous query is a unique entity that is processed by
> > servers independently. With your example, the server node will
> > execute all 20 filters for every cache insert/update operation. The
> > server will notify through local listeners only those clients whose
> > remote filters returned 'true'.
> >
> > -
> > Denis
> >
> >
> > On Thu, Jun 4, 2020 at 8:44 PM <Ro...@t-systems.com> wrote:
> >
> > > Hi Community,
> > >
> > > I ask this question here because I haven't found the answer in the
> > > documentation.
> > >
> > > Could you please clarify how Continuous Queries work? What the
> > > behavior of Continuous Queries if we have several clients with
> > > different Remote Filters on the same cache? For example, if we have:
> > > one server node with cache and we have up to 20 client nodes each
> > > of them will execute Continuous Query on the same cache but with
> > > different Remote Filters. Will each client get the data according
> > > to its remote filter? Or it is supposed to have only one Remote
> > > Filter for all clients and every client should filter data in its
> > > local event
> > listener?
> > > I would be grateful if you send some link which describes the
> > > behavior of Continuous Queries more thoroughly.
> > > Best regards,
> > > Roman
> > >
> >
>
Re: Continuous Queries with several remote filter on the same cache
Posted by Denis Magda <dm...@apache.org>.
Roman,
The updates are ordered per partition. Let's take this example of an
application updating several records:
put (k1, val1) => mapped to partition_10 => node_A
put (k2, val2) => mapped to partition_5 => node_B
put (k3, val3) => mapped to partition_10 => node_A
It's guaranteed that a continuous query listener will be notified about k1
and k3 updates in this order - k1 first and k3 after. As for the k2 update,
it can arrive at any time (i.e., before k1, after k3 or in the middle).
-
Denis
On Fri, Jun 26, 2020 at 12:58 AM <Ro...@t-systems.com> wrote:
> Hi Denis,
>
> Thanks! Is there some guarantee about the order of the updates? Even when
> we have multiple cache nodes.
>
> Best regards,
> Roman
>
> -----Original Message-----
> From: Denis Magda <dm...@apache.org>
> Sent: Monday, June 8, 2020 10:20 PM
> To: dev <de...@ignite.apache.org>
> Subject: Re: Continuous Queries with several remote filter on the same
> cache
>
> Roman,
>
> Please check the following methods:
> * CacheContiniousQueryHandler (the filter usage):
>
> https://github.com/apache/ignite/blob/6955ac291352dd67c1f84a006cda512ee54f38bb/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java#L994
> * CacheContinuousQueryManager (the listener execution):
>
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryManager.java#L376
>
> -
> Denis
>
>
> On Sun, Jun 7, 2020 at 12:19 AM <Ro...@t-systems.com> wrote:
>
> > Hi Denis,
> > A big thank you for the answer.
> > Could you please tell me where can I find this logic in the sources.
> > Which package should I look into?
> >
> > -----Original Message-----
> > From: Denis Magda <dm...@apache.org>
> > Sent: Saturday, June 6, 2020 2:07 AM
> > To: dev <de...@ignite.apache.org>
> > Subject: Re: Continuous Queries with several remote filter on the same
> > cache
> >
> > Hi Roman,
> >
> > Every continuous query is a unique entity that is processed by servers
> > independently. With your example, the server node will execute all 20
> > filters for every cache insert/update operation. The server will
> > notify through local listeners only those clients whose remote filters
> > returned 'true'.
> >
> > -
> > Denis
> >
> >
> > On Thu, Jun 4, 2020 at 8:44 PM <Ro...@t-systems.com> wrote:
> >
> > > Hi Community,
> > >
> > > I ask this question here because I haven't found the answer in the
> > > documentation.
> > >
> > > Could you please clarify how Continuous Queries work? What the
> > > behavior of Continuous Queries if we have several clients with
> > > different Remote Filters on the same cache? For example, if we have:
> > > one server node with cache and we have up to 20 client nodes each of
> > > them will execute Continuous Query on the same cache but with
> > > different Remote Filters. Will each client get the data according to
> > > its remote filter? Or it is supposed to have only one Remote Filter
> > > for all clients and every client should filter data in its local
> > > event
> > listener?
> > > I would be grateful if you send some link which describes the
> > > behavior of Continuous Queries more thoroughly.
> > > Best regards,
> > > Roman
> > >
> >
>
RE: Continuous Queries with several remote filter on the same cache
Posted by Ro...@t-systems.com.
Hi Denis,
Thanks! Is there some guarantee about the order of the updates? Even when we have multiple cache nodes.
Best regards,
Roman
-----Original Message-----
From: Denis Magda <dm...@apache.org>
Sent: Monday, June 8, 2020 10:20 PM
To: dev <de...@ignite.apache.org>
Subject: Re: Continuous Queries with several remote filter on the same cache
Roman,
Please check the following methods:
* CacheContiniousQueryHandler (the filter usage):
https://github.com/apache/ignite/blob/6955ac291352dd67c1f84a006cda512ee54f38bb/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java#L994
* CacheContinuousQueryManager (the listener execution):
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryManager.java#L376
-
Denis
On Sun, Jun 7, 2020 at 12:19 AM <Ro...@t-systems.com> wrote:
> Hi Denis,
> A big thank you for the answer.
> Could you please tell me where can I find this logic in the sources.
> Which package should I look into?
>
> -----Original Message-----
> From: Denis Magda <dm...@apache.org>
> Sent: Saturday, June 6, 2020 2:07 AM
> To: dev <de...@ignite.apache.org>
> Subject: Re: Continuous Queries with several remote filter on the same
> cache
>
> Hi Roman,
>
> Every continuous query is a unique entity that is processed by servers
> independently. With your example, the server node will execute all 20
> filters for every cache insert/update operation. The server will
> notify through local listeners only those clients whose remote filters
> returned 'true'.
>
> -
> Denis
>
>
> On Thu, Jun 4, 2020 at 8:44 PM <Ro...@t-systems.com> wrote:
>
> > Hi Community,
> >
> > I ask this question here because I haven't found the answer in the
> > documentation.
> >
> > Could you please clarify how Continuous Queries work? What the
> > behavior of Continuous Queries if we have several clients with
> > different Remote Filters on the same cache? For example, if we have:
> > one server node with cache and we have up to 20 client nodes each of
> > them will execute Continuous Query on the same cache but with
> > different Remote Filters. Will each client get the data according to
> > its remote filter? Or it is supposed to have only one Remote Filter
> > for all clients and every client should filter data in its local
> > event
> listener?
> > I would be grateful if you send some link which describes the
> > behavior of Continuous Queries more thoroughly.
> > Best regards,
> > Roman
> >
>
Re: Continuous Queries with several remote filter on the same cache
Posted by Denis Magda <dm...@apache.org>.
Roman,
Please check the following methods:
* CacheContiniousQueryHandler (the filter usage):
https://github.com/apache/ignite/blob/6955ac291352dd67c1f84a006cda512ee54f38bb/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java#L994
* CacheContinuousQueryManager (the listener execution):
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryManager.java#L376
-
Denis
On Sun, Jun 7, 2020 at 12:19 AM <Ro...@t-systems.com> wrote:
> Hi Denis,
> A big thank you for the answer.
> Could you please tell me where can I find this logic in the sources. Which
> package should I look into?
>
> -----Original Message-----
> From: Denis Magda <dm...@apache.org>
> Sent: Saturday, June 6, 2020 2:07 AM
> To: dev <de...@ignite.apache.org>
> Subject: Re: Continuous Queries with several remote filter on the same
> cache
>
> Hi Roman,
>
> Every continuous query is a unique entity that is processed by servers
> independently. With your example, the server node will execute all 20
> filters for every cache insert/update operation. The server will notify
> through local listeners only those clients whose remote filters returned
> 'true'.
>
> -
> Denis
>
>
> On Thu, Jun 4, 2020 at 8:44 PM <Ro...@t-systems.com> wrote:
>
> > Hi Community,
> >
> > I ask this question here because I haven't found the answer in the
> > documentation.
> >
> > Could you please clarify how Continuous Queries work? What the
> > behavior of Continuous Queries if we have several clients with
> > different Remote Filters on the same cache? For example, if we have:
> > one server node with cache and we have up to 20 client nodes each of
> > them will execute Continuous Query on the same cache but with
> > different Remote Filters. Will each client get the data according to
> > its remote filter? Or it is supposed to have only one Remote Filter
> > for all clients and every client should filter data in its local event
> listener?
> > I would be grateful if you send some link which describes the behavior
> > of Continuous Queries more thoroughly.
> > Best regards,
> > Roman
> >
>
RE: Continuous Queries with several remote filter on the same cache
Posted by Ro...@t-systems.com.
Hi Denis,
A big thank you for the answer.
Could you please tell me where can I find this logic in the sources. Which package should I look into?
-----Original Message-----
From: Denis Magda <dm...@apache.org>
Sent: Saturday, June 6, 2020 2:07 AM
To: dev <de...@ignite.apache.org>
Subject: Re: Continuous Queries with several remote filter on the same cache
Hi Roman,
Every continuous query is a unique entity that is processed by servers independently. With your example, the server node will execute all 20 filters for every cache insert/update operation. The server will notify through local listeners only those clients whose remote filters returned 'true'.
-
Denis
On Thu, Jun 4, 2020 at 8:44 PM <Ro...@t-systems.com> wrote:
> Hi Community,
>
> I ask this question here because I haven't found the answer in the
> documentation.
>
> Could you please clarify how Continuous Queries work? What the
> behavior of Continuous Queries if we have several clients with
> different Remote Filters on the same cache? For example, if we have:
> one server node with cache and we have up to 20 client nodes each of
> them will execute Continuous Query on the same cache but with
> different Remote Filters. Will each client get the data according to
> its remote filter? Or it is supposed to have only one Remote Filter
> for all clients and every client should filter data in its local event listener?
> I would be grateful if you send some link which describes the behavior
> of Continuous Queries more thoroughly.
> Best regards,
> Roman
>
Re: Continuous Queries with several remote filter on the same cache
Posted by Denis Magda <dm...@apache.org>.
Hi Roman,
Every continuous query is a unique entity that is processed by servers
independently. With your example, the server node will execute all 20
filters for every cache insert/update operation. The server will notify
through local listeners only those clients whose remote filters returned
'true'.
-
Denis
On Thu, Jun 4, 2020 at 8:44 PM <Ro...@t-systems.com> wrote:
> Hi Community,
>
> I ask this question here because I haven't found the answer in the
> documentation.
>
> Could you please clarify how Continuous Queries work? What the behavior of
> Continuous Queries if we have several clients with different Remote Filters
> on the same cache? For example, if we have: one server node with cache and
> we have up to 20 client nodes each of them will execute Continuous Query on
> the same cache but with different Remote Filters. Will each client get the
> data according to its remote filter? Or it is supposed to have only one
> Remote Filter for all clients and every client should filter data in its
> local event listener?
> I would be grateful if you send some link which describes the behavior of
> Continuous Queries more thoroughly.
> Best regards,
> Roman
>