You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ignite.apache.org by Jigna <ji...@hatch.com> on 2021/01/18 16:41:47 UTC

Event Listener on Server Node

Hi Team,

I am using Ignite through Gridgain community version. Our application is in
c#/.net. We have deployed our application on k8s cluster. 

Our server node generates the cache events and I want to write a listener to
listen to them. But, Ignite c# API does not support remote listening. So I
can not write the listener on my client node for cache events, which are
generated on the server. Now, if I want to listen the events, I want to
write a listener on server only. But, I am not sure how to register my
listener on the server node. E.g.

 events.LocalListen(new LocalListener(), EventType.CacheObjectPut,
EventType.CacheObjectRead,
        EventType.CacheObjectRemoved);

I can change the server Ignite configuration to enable the event types, but
can I register my listener in the configuration?

<property name="includeEventTypes">
            <list>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_NODE_LEFT"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_NODE_JOINED"/>
            </list>
        </property>

Would you please help me to register my event listener on the server node? 

Thanks,
Jigna.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Event Listener on Server Node

Posted by Pavel Tupitsyn <pt...@apache.org>.

You have to start a continuous query for every cache separately.

On Thu, Jan 21, 2021 at 1:47 AM Jigna <ji...@hatch.com> wrote:

> Hi,
>
> Thank you for sharing the solution. It really helped me to understand the
> continuous queries for remote event listening. I have another question for
> continuous queries.
>
> Once I write the below code, then it will listen the events for
> "cache-name"
> cache.
>
> var cache = ignite.GetOrCreateCache<int, string>("cache-name");
> cache.QueryContinuous(new ContinuousQuery<int, string>(new
> EventListener()));
>
> If I will create a new cache after that, this query would not listen for
> those cache events, which I can understand.
>
> As I mentioned, we have multiple projects and multiple caches. Once I will
> register my continuous queries, it should listen for all the caches( for
> newly added as well). I am not sure how to write my continuous query to
> support all the caches. Would you please guide me?
>
> Thanks,
> Jigna
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Event Listener on Server Node

Posted by Jigna <ji...@hatch.com>.

Hi,

Thank you for sharing the solution. It really helped me to understand the
continuous queries for remote event listening. I have another question for
continuous queries.

Once I write the below code, then it will listen the events for "cache-name"
cache.

var cache = ignite.GetOrCreateCache<int, string>("cache-name");
cache.QueryContinuous(new ContinuousQuery<int, string>(new
EventListener()));

If I will create a new cache after that, this query would not listen for
those cache events, which I can understand. 

As I mentioned, we have multiple projects and multiple caches. Once I will
register my continuous queries, it should listen for all the caches( for
newly added as well). I am not sure how to write my continuous query to
support all the caches. Would you please guide me?

Thanks, 
Jigna







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Event Listener on Server Node

Posted by Pavel Tupitsyn <pt...@apache.org>.

Continuous Query supports both local and remote cache events, in .NET too.
Try it - create two simple programs and run them both:

// Program 1:
            var ignite = Ignition.Start();
            var cache = ignite.GetOrCreateCache<int, string>("cache-name");

            for (var i = 0; i < int.MaxValue; i++)
            {
                Console.WriteLine($"Adding {i}");
                cache[i] = $"Hello {i}!";
                Thread.Sleep(500);
            }

// Program 2:
        var ignite = Ignition.Start();
        var cache = ignite.GetOrCreateCache<int, string>("cache-name");
        cache.QueryContinuous(new ContinuousQuery<int, string>(new
EventListener()));
        Console.ReadKey();

        class EventListener : ICacheEntryEventListener<int, string>
        {
            public void OnEvent(IEnumerable<ICacheEntryEvent<int, string>>
evts)
            {
                foreach (var evt in evts)
                {
                    Console.WriteLine(evt.Value);
                }
            }
        }

On Wed, Jan 20, 2021 at 1:15 AM Jigna <ji...@hatch.com> wrote:

> Hi,
>
> Now I am writing continuous queries for my scenario and I want to listen
> server node events as per my requirement.
>
> Ignite .net API does not support listening for remote events.
>
> As per my understanding, I have server node, which generates all the cache
> events. So, when I write continuous query, I need to write it in a separate
> application, it will start as a client node and listen the events using
> local listener(client node events). But, I want to listen the remote
> events(server node events) in the continuous query application.
>
> I am not sure how to support remote event listening in .net Ignite
> application. Would you please verify my understanding?
>
> I am not sure about my solution. Please suggest me how to write continuous
> query and listen server cache events in .Net Ignite application.
>
> Thank you,
> Jigna
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Event Listener on Server Node

Posted by Jigna <ji...@hatch.com>.

Hi,

Now I am writing continuous queries for my scenario and I want to listen
server node events as per my requirement.

Ignite .net API does not support listening for remote events. 

As per my understanding, I have server node, which generates all the cache
events. So, when I write continuous query, I need to write it in a separate
application, it will start as a client node and listen the events using
local listener(client node events). But, I want to listen the remote
events(server node events) in the continuous query application. 

I am not sure how to support remote event listening in .net Ignite
application. Would you please verify my understanding?

I am not sure about my solution. Please suggest me how to write continuous
query and listen server cache events in .Net Ignite application.

Thank you,
Jigna



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Event Listener on Server Node

Posted by Pavel Tupitsyn <pt...@apache.org>.

In my experience, Continuous Query is a better approach than Events.

On Tue, Jan 19, 2021 at 9:39 PM Jigna <ji...@hatch.com> wrote:

> Hi,
>
> We use many caches because we have different schemas.
>
> Basically, cache2 is a consolidation from different caches. This
> consolidation only takes some attributes from cache 1 to cache 2 then
> others
> from cache 3 to cache 2. can I achieve this consolidation better using
> events or continuous query?
>
> Thanks,
> Jigna
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Event Listener on Server Node

Posted by Jigna <ji...@hatch.com>.

Hi,

We use many caches because we have different schemas.

Basically, cache2 is a consolidation from different caches. This
consolidation only takes some attributes from cache 1 to cache 2 then others
from cache 3 to cache 2. can I achieve this consolidation better using
events or continuous query?

Thanks,
Jigna



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Event Listener on Server Node

Posted by Pavel Tupitsyn <pt...@apache.org>.

> we will have hundreds to thousands of caches

This is not a very good idea [1], every cache imposes some overhead,
and starting/stopping caches is not cheap as well.

Please consider a different approach where data is grouped into a smaller
number of caches.

> When we insert a record in cache1, then we want to add/update that record
into cache2

What's the idea behind that? Can you keep all the data for the given record
in one cache instead of multiple?

[1]
https://stackoverflow.com/questions/55297333/does-apache-ignite-support-10-thousands-of-cache

On Tue, Jan 19, 2021 at 5:46 PM Jigna <ji...@hatch.com> wrote:

> Hi,
>
> In addition to that, We have so many projects. So, we will have hundreds to
> thousands of caches at any given time and that the number changes as we add
> and remove projects.
>
> I am not sure in this scenario, which solution suits best, continuous query
> or events handling in a service? Would you please help me to understand?
>
> Thanks,
> Jigna
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Event Listener on Server Node

Posted by Jigna <ji...@hatch.com>.

Hi,

In addition to that, We have so many projects. So, we will have hundreds to
thousands of caches at any given time and that the number changes as we add
and remove projects. 

I am not sure in this scenario, which solution suits best, continuous query
or events handling in a service? Would you please help me to understand?

Thanks,
Jigna





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Event Listener on Server Node

Posted by Jigna <ji...@hatch.com>.

Hi,

That works perfect. Thank you for the solution.

Jigna.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Event Listener on Server Node

Posted by Pavel Tupitsyn <pt...@apache.org>.

Continuous Queries [1] are designed specifically for use cases like that -
to handle cache updates efficiently, please use them instead of Events.

You can subscribe to local and remote cache updates, and define a remote
filter if necessary.

[1] https://ignite.apache.org/docs/latest/key-value-api/continuous-queries

On Mon, Jan 18, 2021 at 9:44 PM Jigna <ji...@hatch.com> wrote:

> Hi Pavel,
>
> I really appreciate your help. I will try to explain my scenario:
>
> 1. When we insert a record in cache1, then we want to add/update that
> record
> into cache2.
> 2. For that, we want to listen the cache event and the listener will
> identify the data and either inserts or updates that record into cache2.
>
> Similarly, we have other caches like cache3 and cache4. So when we insert
> the record into cache3 or cache4, then we want to insert/update that record
> into cache2.
>
> Now, the cache events are generated on the server node and I want to write
> a
> listener service on the server node, which will run the above logic. We use
> c# for our development.
>
> What should I do in this case? Would you please help me?
>
> Thank you,
> Jigna.
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Event Listener on Server Node

Posted by Jigna <ji...@hatch.com>.

Hi Pavel,

I really appreciate your help. I will try to explain my scenario:

1. When we insert a record in cache1, then we want to add/update that record
into cache2.
2. For that, we want to listen the cache event and the listener will
identify the data and either inserts or updates that record into cache2.

Similarly, we have other caches like cache3 and cache4. So when we insert
the record into cache3 or cache4, then we want to insert/update that record
into cache2. 

Now, the cache events are generated on the server node and I want to write a
listener service on the server node, which will run the above logic. We use
c# for our development. 

What should I do in this case? Would you please help me?

Thank you,
Jigna.






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Event Listener on Server Node

Posted by Pavel Tupitsyn <pt...@apache.org>.

Hi,

You can use a combination of LocalListen and Services, Messaging or Compute
to achieve something similar to RemoteListen, depending on the use case.

If you explain the use case in more detail, I can suggest a specific
approach.

Thanks,
Pavel

On Mon, Jan 18, 2021 at 7:42 PM Jigna <ji...@hatch.com> wrote:

> Hi Team,
>
> I am using Ignite through Gridgain community version. Our application is in
> c#/.net. We have deployed our application on k8s cluster.
>
> Our server node generates the cache events and I want to write a listener
> to
> listen to them. But, Ignite c# API does not support remote listening. So I
> can not write the listener on my client node for cache events, which are
> generated on the server. Now, if I want to listen the events, I want to
> write a listener on server only. But, I am not sure how to register my
> listener on the server node. E.g.
>
>  events.LocalListen(new LocalListener(), EventType.CacheObjectPut,
> EventType.CacheObjectRead,
>         EventType.CacheObjectRemoved);
>
> I can change the server Ignite configuration to enable the event types, but
> can I register my listener in the configuration?
>
> <property name="includeEventTypes">
>             <list>
>                 <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
>                 <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
>                 <util:constant
>
> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
>                 <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_NODE_LEFT"/>
>                 <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_NODE_JOINED"/>
>             </list>
>         </property>
>
> Would you please help me to register my event listener on the server node?
>
> Thanks,
> Jigna.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>