You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by SunSatION <su...@gmail.com> on 2019/10/16 14:04:45 UTC
Action performed multiple times when using HA ignite clients using
continuous queries
Hi,
We have a scenario where we're using Ignite Kafka Datastream to write to
cache which are setup using PARTITIONED affinity. We're using continuous
queries to identify changes to each cache entry and perform an action if the
entry changes which are running on the client. We have 3 server nodes and 3
client nodes for HA. The problem is that all clients are receiving the cache
changes and the action is performed 3 times, once for each client. We are
unable to use the affinity key of the cache entry as it returns the id of
the server nodes.
Any ideas on the a way that the action is performed once without the client
nodes knowing of each other?
Thanks,
Dorian
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Action performed multiple times when using HA ignite clients
using continuous queries
Posted by Stephen Darlington <st...@gridgain.com>.
You can deploy services to client nodes, for example:
val service = ignite.services(ignite.cluster().forClients())
service.deployClusterSingleton("TestService", new TestServiceImpl())
If you want to have independent clients, as you currently have, I suspect you’d have to implement your own fail over mechanism.
Regards,
Stephen
> On 17 Oct 2019, at 10:37, SunSatION <su...@gmail.com> wrote:
>
> I don't wish to deploy the client code onto each and every server as we would like to keep the separation between business logic and data. Additionally, we are pushing the changes to Kafka and therefore not sure if it's heavy to perform the push the Service grid
>
> On Wed, Oct 16, 2019 at 6:02 PM Stephen Darlington <stephen.darlington@gridgain.com <ma...@gridgain.com>> wrote:
> Your clients don’t “know” about each other, so, yes, they’re going to duplicate work. Could you use the Service Grid to run your client? That way it would fail over automatically.
>
> Regards,
> Stephen
>
> > On 16 Oct 2019, at 16:04, SunSatION <sunsation.xsoft@gmail.com <ma...@gmail.com>> wrote:
> >
> > Hi,
> >
> > We have a scenario where we're using Ignite Kafka Datastream to write to
> > cache which are setup using PARTITIONED affinity. We're using continuous
> > queries to identify changes to each cache entry and perform an action if the
> > entry changes which are running on the client. We have 3 server nodes and 3
> > client nodes for HA. The problem is that all clients are receiving the cache
> > changes and the action is performed 3 times, once for each client. We are
> > unable to use the affinity key of the cache entry as it returns the id of
> > the server nodes.
> >
> > Any ideas on the a way that the action is performed once without the client
> > nodes knowing of each other?
> >
> > Thanks,
> > Dorian
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ <http://apache-ignite-users.70518.x6.nabble.com/>
>
>
>
>
> --
> Two things are infinite: the universe and the human stupidity; and I'm not sure about the universe - Albert Einstein
Re: Action performed multiple times when using HA ignite clients
using continuous queries
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
You can use distributed locks to make sure only one client is working on a
chunk of data.
Regards,
--
Ilya Kasnacheev
чт, 17 окт. 2019 г. в 12:37, SunSatION <su...@gmail.com>:
> I don't wish to deploy the client code onto each and every server as we
> would like to keep the separation between business logic and data.
> Additionally, we are pushing the changes to Kafka and therefore not sure if
> it's heavy to perform the push the Service grid
>
> On Wed, Oct 16, 2019 at 6:02 PM Stephen Darlington <
> stephen.darlington@gridgain.com> wrote:
>
>> Your clients don’t “know” about each other, so, yes, they’re going to
>> duplicate work. Could you use the Service Grid to run your client? That way
>> it would fail over automatically.
>>
>> Regards,
>> Stephen
>>
>> > On 16 Oct 2019, at 16:04, SunSatION <su...@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > We have a scenario where we're using Ignite Kafka Datastream to write to
>> > cache which are setup using PARTITIONED affinity. We're using continuous
>> > queries to identify changes to each cache entry and perform an action
>> if the
>> > entry changes which are running on the client. We have 3 server nodes
>> and 3
>> > client nodes for HA. The problem is that all clients are receiving the
>> cache
>> > changes and the action is performed 3 times, once for each client. We
>> are
>> > unable to use the affinity key of the cache entry as it returns the id
>> of
>> > the server nodes.
>> >
>> > Any ideas on the a way that the action is performed once without the
>> client
>> > nodes knowing of each other?
>> >
>> > Thanks,
>> > Dorian
>> >
>> >
>> >
>> > --
>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>>
>>
>
> --
> Two things are infinite: the universe and the human stupidity; and I'm not
> sure about the universe - Albert Einstein
>
Re: Action performed multiple times when using HA ignite clients
using continuous queries
Posted by SunSatION <su...@gmail.com>.
I don't wish to deploy the client code onto each and every server as we
would like to keep the separation between business logic and data.
Additionally, we are pushing the changes to Kafka and therefore not sure if
it's heavy to perform the push the Service grid
On Wed, Oct 16, 2019 at 6:02 PM Stephen Darlington <
stephen.darlington@gridgain.com> wrote:
> Your clients don’t “know” about each other, so, yes, they’re going to
> duplicate work. Could you use the Service Grid to run your client? That way
> it would fail over automatically.
>
> Regards,
> Stephen
>
> > On 16 Oct 2019, at 16:04, SunSatION <su...@gmail.com> wrote:
> >
> > Hi,
> >
> > We have a scenario where we're using Ignite Kafka Datastream to write to
> > cache which are setup using PARTITIONED affinity. We're using continuous
> > queries to identify changes to each cache entry and perform an action if
> the
> > entry changes which are running on the client. We have 3 server nodes
> and 3
> > client nodes for HA. The problem is that all clients are receiving the
> cache
> > changes and the action is performed 3 times, once for each client. We are
> > unable to use the affinity key of the cache entry as it returns the id of
> > the server nodes.
> >
> > Any ideas on the a way that the action is performed once without the
> client
> > nodes knowing of each other?
> >
> > Thanks,
> > Dorian
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>
--
Two things are infinite: the universe and the human stupidity; and I'm not
sure about the universe - Albert Einstein
Re: Action performed multiple times when using HA ignite clients
using continuous queries
Posted by Stephen Darlington <st...@gridgain.com>.
Your clients don’t “know” about each other, so, yes, they’re going to duplicate work. Could you use the Service Grid to run your client? That way it would fail over automatically.
Regards,
Stephen
> On 16 Oct 2019, at 16:04, SunSatION <su...@gmail.com> wrote:
>
> Hi,
>
> We have a scenario where we're using Ignite Kafka Datastream to write to
> cache which are setup using PARTITIONED affinity. We're using continuous
> queries to identify changes to each cache entry and perform an action if the
> entry changes which are running on the client. We have 3 server nodes and 3
> client nodes for HA. The problem is that all clients are receiving the cache
> changes and the action is performed 3 times, once for each client. We are
> unable to use the affinity key of the cache entry as it returns the id of
> the server nodes.
>
> Any ideas on the a way that the action is performed once without the client
> nodes knowing of each other?
>
> Thanks,
> Dorian
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/