Get geode region insert completion event

[aashish choudhary <aa...@gmail.com>]

Use case description:-

We have requirement where in we will have two geode regions let's say
position1 and position2 now position1 will have positions as of 25
september after end of the day a data load operation will run which will
put data to position2 region. At this point while data load is running any
get request will be served from position1 region because there is a
possibility of getting stale data. Now the client will have to make a
switch to position2 region to serve get request with latest data after the
load gets completed. So is there any way by which a client application
could make a switch to different region once the data load gets completed .


With best regards,
Ashish

Re: Get geode region insert completion event

[Anthony Baker <ab...@pivotal.io>]

I’m sure there are lots of patterns that could be used for this.  Here’s one idea:  create a third region with a key that points to the region with the current data.  Use a CQ or subscription or poll to get the key pointing to the region with current data.  Update the reference key to point to the other region after the data load is done.

Anthony


> On Sep 26, 2018, at 1:40 AM, aashish choudhary <aa...@gmail.com> wrote:
> 
> Use case description:-
> 
> We have requirement where in we will have two geode regions let's say position1 and position2 now position1 will have positions as of 25 september after end of the day a data load operation will run which will put data to position2 region. At this point while data load is running any get request will be served from position1 region because there is a possibility of getting stale data. Now the client will have to make a switch to position2 region to serve get request with latest data after the load gets completed. So is there any way by which a client application could make a switch to different region once the data load gets completed . 
> 
> 
> With best regards,
> Ashish

Re: Get geode region insert completion event

[aashish choudhary <aa...@gmail.com>]

Thanks Dharam!!!
On another note how do you handle failure of CQ in your case. I have
started reading about CQ and it does give some api to check CQ state. Do
you handle it in similar way?

Ashish


On Thu 27 Sep, 2018, 1:36 PM Thacker, Dharam, <dh...@jpmorgan.com>
wrote:

> That’s correct Aashish!
>
>
>
> Class à DataLoadEvent { String current; //getters + setters }
>
> Region<UUID,DataLoadEvent> à  /Event
>
>
>
> CQ : select e.current from /Event e
>
>
>
> As soon as you consume it in CQHandler, remove it from /Event region.
>
>
>
> Thanks,
>
> Dharam
>
>
>
>
>
> *From:* aashish choudhary [mailto:aashish.choudhary1@gmail.com]
> *Sent:* Thursday, September 27, 2018 1:24 PM
> *To:* user@geode.apache.org
> *Subject:* Re: Get geode region insert completion event
>
>
>
> Thanks Mike, Anthony, Anil, Dharam for your inputs.Must say we have great
> community support for geode.
>
>
>
> Dharam,.
>
> I am trying to understand the part where in you said that "Now as soon as
> data load on position2 is completed you publish a 'DataLoadCompleted' event
> into new /event region with attribute as current=position2". So with this
> event we basically mean that we will put and entry in the /event region.
> And then from there on CQ will take care of notifying it to client to make
> the switch as you suggested.
>
>
> Ashish
>
>
>
> On Wed 26 Sep, 2018, 11:32 PM Dharam Thacker, <dh...@gmail.com>
> wrote:
>
> If my understanding is correct, let me try to explain how I have done for
> my use case.
>
>
>
> Let's say you have position1 and position2 regions on server.
>
>
>
> - Client is accessing data from position1.
>
> - On end of day you receive a trigger and data will be loaded in position2
> region.
>
> - Meanwhile data load is running, client is still accessing data from
> position1.
>
>
>
> Now as soon as data load on position2 is completed you publish a
> 'DataLoadCompleted' event into new /event region with attribute as
> current=position2.
>
>
>
> Let your client register cqlistsner and on this match event you switch to
> position2 region in cqlistsner handler.
>
>
>
> I am assuming that in your client you already have reference to both
> client proxy region or gemfire template.
>
>
>
> Example:
>
>
>
> GemfireTemplate position1Template
>
> GemfireTemplate position2Template
>
>
>
> GemfireTemplate currentTemplate
>
>
>
> On cqlistsner event you switch current template to position1 or position2
> depending on "current" property explained above.
>
>
>
> Thanks,
>
> Dharam
>
>
>
> On Wed, Sep 26, 2018, 23:17 Michael Stolz <ms...@pivotal.io> wrote:
>
> Then you would have to check that the pointer record is pointing to the
> region you fetched it from. If not, redo the operation on the region it IS
> pointing to.
>
>
> --
>
> Mike Stolz
>
> Principal Engineer, GemFire Product Lead
>
> Mobile: +1-631-835-4771
>
>
>
>
>
>
>
> On Wed, Sep 26, 2018 at 1:47 PM Michael Stolz <ms...@pivotal.io> wrote:
>
> You could store the pointer record in both regions and make sure to
> replace the stale region first then the live region. Then you could use
> getAll( ) including the key of the pointer record as well as the record you
> want.
>
>
> --
>
> Mike Stolz
>
> Principal Engineer, GemFire Product Lead
>
> Mobile: +1-631-835-4771
>
>
>
>
>
>
>
> On Wed, Sep 26, 2018 at 1:43 PM Anilkumar Gingade <ag...@pivotal.io>
> wrote:
>
> Currently, there is no option to specify multiple region in a single API.
> Application has to handle this.
>
> E.g.: One way by doing get() on position2 first, if not found then on
> position1 (it can use functions).
>
> There could be a window where the position2 is getting updated but its not
> yet reflected in the region. Depending on use-case requirement (if its fine
> to get old data when its getting updated), application has to handle it.
>
>
>
> -Anil.
>
>
>
>
>
> On Tue, Sep 25, 2018 at 10:40 PM aashish choudhary <
> aashish.choudhary1@gmail.com> wrote:
>
> Use case description:-
>
>
>
> We have requirement where in we will have two geode regions let's say
> position1 and position2 now position1 will have positions as of 25
> september after end of the day a data load operation will run which will
> put data to position2 region. At this point while data load is running any
> get request will be served from position1 region because there is a
> possibility of getting stale data. Now the client will have to make a
> switch to position2 region to serve get request with latest data after the
> load gets completed. So is there any way by which a client application
> could make a switch to different region once the data load gets completed .
>
>
>
>
>
> With best regards,
> Ashish
>
> This message is confidential and subject to terms at: http://
> www.jpmorgan.com/emaildisclaimer including on confidentiality, legal
> privilege, viruses and monitoring of electronic messages. If you are not
> the intended recipient, please delete this message and notify the sender
> immediately. Any unauthorized use is strictly prohibited.
>

RE: Get geode region insert completion event

["Thacker, Dharam" <dh...@jpmorgan.com>]

That’s correct Aashish!

Class --> DataLoadEvent { String current; //getters + setters }
Region<UUID,DataLoadEvent> -->  /Event

CQ : select e.current from /Event e

As soon as you consume it in CQHandler, remove it from /Event region.

Thanks,
Dharam


From: aashish choudhary [mailto:aashish.choudhary1@gmail.com]
Sent: Thursday, September 27, 2018 1:24 PM
To: user@geode.apache.org
Subject: Re: Get geode region insert completion event

Thanks Mike, Anthony, Anil, Dharam for your inputs.Must say we have great community support for geode.

Dharam,.
I am trying to understand the part where in you said that "Now as soon as data load on position2 is completed you publish a 'DataLoadCompleted' event into new /event region with attribute as current=position2". So with this event we basically mean that we will put and entry in the /event region. And then from there on CQ will take care of notifying it to client to make the switch as you suggested.

Ashish

On Wed 26 Sep, 2018, 11:32 PM Dharam Thacker, <dh...@gmail.com>> wrote:
If my understanding is correct, let me try to explain how I have done for my use case.

Let's say you have position1 and position2 regions on server.

- Client is accessing data from position1.
- On end of day you receive a trigger and data will be loaded in position2 region.
- Meanwhile data load is running, client is still accessing data from position1.

Now as soon as data load on position2 is completed you publish a 'DataLoadCompleted' event into new /event region with attribute as current=position2.

Let your client register cqlistsner and on this match event you switch to position2 region in cqlistsner handler.

I am assuming that in your client you already have reference to both client proxy region or gemfire template.

Example:

GemfireTemplate position1Template
GemfireTemplate position2Template

GemfireTemplate currentTemplate

On cqlistsner event you switch current template to position1 or position2 depending on "current" property explained above.

Thanks,
Dharam

On Wed, Sep 26, 2018, 23:17 Michael Stolz <ms...@pivotal.io>> wrote:
Then you would have to check that the pointer record is pointing to the region you fetched it from. If not, redo the operation on the region it IS pointing to.

--
Mike Stolz
Principal Engineer, GemFire Product Lead
Mobile: +1-631-835-4771



On Wed, Sep 26, 2018 at 1:47 PM Michael Stolz <ms...@pivotal.io>> wrote:
You could store the pointer record in both regions and make sure to replace the stale region first then the live region. Then you could use getAll( ) including the key of the pointer record as well as the record you want.

--
Mike Stolz
Principal Engineer, GemFire Product Lead
Mobile: +1-631-835-4771



On Wed, Sep 26, 2018 at 1:43 PM Anilkumar Gingade <ag...@pivotal.io>> wrote:
Currently, there is no option to specify multiple region in a single API. Application has to handle this.
E.g.: One way by doing get() on position2 first, if not found then on position1 (it can use functions).
There could be a window where the position2 is getting updated but its not yet reflected in the region. Depending on use-case requirement (if its fine to get old data when its getting updated), application has to handle it.

-Anil.


On Tue, Sep 25, 2018 at 10:40 PM aashish choudhary <aa...@gmail.com>> wrote:
Use case description:-

We have requirement where in we will have two geode regions let's say position1 and position2 now position1 will have positions as of 25 september after end of the day a data load operation will run which will put data to position2 region. At this point while data load is running any get request will be served from position1 region because there is a possibility of getting stale data. Now the client will have to make a switch to position2 region to serve get request with latest data after the load gets completed. So is there any way by which a client application could make a switch to different region once the data load gets completed .


With best regards,
Ashish

This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.

Re: Get geode region insert completion event

[aashish choudhary <aa...@gmail.com>]

Thanks Mike, Anthony, Anil, Dharam for your inputs.Must say we have great
community support for geode.

Dharam,.
I am trying to understand the part where in you said that "Now as soon as
data load on position2 is completed you publish a 'DataLoadCompleted' event
into new /event region with attribute as current=position2". So with this
event we basically mean that we will put and entry in the /event region.
And then from there on CQ will take care of notifying it to client to make
the switch as you suggested.

Ashish

On Wed 26 Sep, 2018, 11:32 PM Dharam Thacker, <dh...@gmail.com>
wrote:

> If my understanding is correct, let me try to explain how I have done for
> my use case.
>
> Let's say you have position1 and position2 regions on server.
>
> - Client is accessing data from position1.
> - On end of day you receive a trigger and data will be loaded in position2
> region.
> - Meanwhile data load is running, client is still accessing data from
> position1.
>
> Now as soon as data load on position2 is completed you publish a
> 'DataLoadCompleted' event into new /event region with attribute as
> current=position2.
>
> Let your client register cqlistsner and on this match event you switch to
> position2 region in cqlistsner handler.
>
> I am assuming that in your client you already have reference to both
> client proxy region or gemfire template.
>
> Example:
>
> GemfireTemplate position1Template
> GemfireTemplate position2Template
>
> GemfireTemplate currentTemplate
>
> On cqlistsner event you switch current template to position1 or position2
> depending on "current" property explained above.
>
> Thanks,
> Dharam
>
> On Wed, Sep 26, 2018, 23:17 Michael Stolz <ms...@pivotal.io> wrote:
>
>> Then you would have to check that the pointer record is pointing to the
>> region you fetched it from. If not, redo the operation on the region it IS
>> pointing to.
>>
>> --
>> Mike Stolz
>> Principal Engineer, GemFire Product Lead
>> Mobile: +1-631-835-4771
>>
>>
>>
>> On Wed, Sep 26, 2018 at 1:47 PM Michael Stolz <ms...@pivotal.io> wrote:
>>
>>> You could store the pointer record in both regions and make sure to
>>> replace the stale region first then the live region. Then you could use
>>> getAll( ) including the key of the pointer record as well as the record you
>>> want.
>>>
>>> --
>>> Mike Stolz
>>> Principal Engineer, GemFire Product Lead
>>> Mobile: +1-631-835-4771
>>>
>>>
>>>
>>> On Wed, Sep 26, 2018 at 1:43 PM Anilkumar Gingade <ag...@pivotal.io>
>>> wrote:
>>>
>>>> Currently, there is no option to specify multiple region in a single
>>>> API. Application has to handle this.
>>>> E.g.: One way by doing get() on position2 first, if not found then on
>>>> position1 (it can use functions).
>>>> There could be a window where the position2 is getting updated but its
>>>> not yet reflected in the region. Depending on use-case requirement (if its
>>>> fine to get old data when its getting updated), application has to handle
>>>> it.
>>>>
>>>> -Anil.
>>>>
>>>>
>>>> On Tue, Sep 25, 2018 at 10:40 PM aashish choudhary <
>>>> aashish.choudhary1@gmail.com> wrote:
>>>>
>>>>> Use case description:-
>>>>>
>>>>> We have requirement where in we will have two geode regions let's say
>>>>> position1 and position2 now position1 will have positions as of 25
>>>>> september after end of the day a data load operation will run which will
>>>>> put data to position2 region. At this point while data load is running any
>>>>> get request will be served from position1 region because there is a
>>>>> possibility of getting stale data. Now the client will have to make a
>>>>> switch to position2 region to serve get request with latest data after the
>>>>> load gets completed. So is there any way by which a client application
>>>>> could make a switch to different region once the data load gets completed .
>>>>>
>>>>>
>>>>> With best regards,
>>>>> Ashish
>>>>>
>>>>

Re: Get geode region insert completion event

[Dharam Thacker <dh...@gmail.com>]

If my understanding is correct, let me try to explain how I have done for
my use case.

Let's say you have position1 and position2 regions on server.

- Client is accessing data from position1.
- On end of day you receive a trigger and data will be loaded in position2
region.
- Meanwhile data load is running, client is still accessing data from
position1.

Now as soon as data load on position2 is completed you publish a
'DataLoadCompleted' event into new /event region with attribute as
current=position2.

Let your client register cqlistsner and on this match event you switch to
position2 region in cqlistsner handler.

I am assuming that in your client you already have reference to both client
proxy region or gemfire template.

Example:

GemfireTemplate position1Template
GemfireTemplate position2Template

GemfireTemplate currentTemplate

On cqlistsner event you switch current template to position1 or position2
depending on "current" property explained above.

Thanks,
Dharam

On Wed, Sep 26, 2018, 23:17 Michael Stolz <ms...@pivotal.io> wrote:

> Then you would have to check that the pointer record is pointing to the
> region you fetched it from. If not, redo the operation on the region it IS
> pointing to.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771
>
>
>
> On Wed, Sep 26, 2018 at 1:47 PM Michael Stolz <ms...@pivotal.io> wrote:
>
>> You could store the pointer record in both regions and make sure to
>> replace the stale region first then the live region. Then you could use
>> getAll( ) including the key of the pointer record as well as the record you
>> want.
>>
>> --
>> Mike Stolz
>> Principal Engineer, GemFire Product Lead
>> Mobile: +1-631-835-4771
>>
>>
>>
>> On Wed, Sep 26, 2018 at 1:43 PM Anilkumar Gingade <ag...@pivotal.io>
>> wrote:
>>
>>> Currently, there is no option to specify multiple region in a single
>>> API. Application has to handle this.
>>> E.g.: One way by doing get() on position2 first, if not found then on
>>> position1 (it can use functions).
>>> There could be a window where the position2 is getting updated but its
>>> not yet reflected in the region. Depending on use-case requirement (if its
>>> fine to get old data when its getting updated), application has to handle
>>> it.
>>>
>>> -Anil.
>>>
>>>
>>> On Tue, Sep 25, 2018 at 10:40 PM aashish choudhary <
>>> aashish.choudhary1@gmail.com> wrote:
>>>
>>>> Use case description:-
>>>>
>>>> We have requirement where in we will have two geode regions let's say
>>>> position1 and position2 now position1 will have positions as of 25
>>>> september after end of the day a data load operation will run which will
>>>> put data to position2 region. At this point while data load is running any
>>>> get request will be served from position1 region because there is a
>>>> possibility of getting stale data. Now the client will have to make a
>>>> switch to position2 region to serve get request with latest data after the
>>>> load gets completed. So is there any way by which a client application
>>>> could make a switch to different region once the data load gets completed .
>>>>
>>>>
>>>> With best regards,
>>>> Ashish
>>>>
>>>

Re: Get geode region insert completion event

[Michael Stolz <ms...@pivotal.io>]

Then you would have to check that the pointer record is pointing to the
region you fetched it from. If not, redo the operation on the region it IS
pointing to.

--
Mike Stolz
Principal Engineer, GemFire Product Lead
Mobile: +1-631-835-4771



On Wed, Sep 26, 2018 at 1:47 PM Michael Stolz <ms...@pivotal.io> wrote:

> You could store the pointer record in both regions and make sure to
> replace the stale region first then the live region. Then you could use
> getAll( ) including the key of the pointer record as well as the record you
> want.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771
>
>
>
> On Wed, Sep 26, 2018 at 1:43 PM Anilkumar Gingade <ag...@pivotal.io>
> wrote:
>
>> Currently, there is no option to specify multiple region in a single API.
>> Application has to handle this.
>> E.g.: One way by doing get() on position2 first, if not found then on
>> position1 (it can use functions).
>> There could be a window where the position2 is getting updated but its
>> not yet reflected in the region. Depending on use-case requirement (if its
>> fine to get old data when its getting updated), application has to handle
>> it.
>>
>> -Anil.
>>
>>
>> On Tue, Sep 25, 2018 at 10:40 PM aashish choudhary <
>> aashish.choudhary1@gmail.com> wrote:
>>
>>> Use case description:-
>>>
>>> We have requirement where in we will have two geode regions let's say
>>> position1 and position2 now position1 will have positions as of 25
>>> september after end of the day a data load operation will run which will
>>> put data to position2 region. At this point while data load is running any
>>> get request will be served from position1 region because there is a
>>> possibility of getting stale data. Now the client will have to make a
>>> switch to position2 region to serve get request with latest data after the
>>> load gets completed. So is there any way by which a client application
>>> could make a switch to different region once the data load gets completed .
>>>
>>>
>>> With best regards,
>>> Ashish
>>>
>>

Re: Get geode region insert completion event

[Michael Stolz <ms...@pivotal.io>]

You could store the pointer record in both regions and make sure to replace
the stale region first then the live region. Then you could use getAll( )
including the key of the pointer record as well as the record you want.

--
Mike Stolz
Principal Engineer, GemFire Product Lead
Mobile: +1-631-835-4771



On Wed, Sep 26, 2018 at 1:43 PM Anilkumar Gingade <ag...@pivotal.io>
wrote:

> Currently, there is no option to specify multiple region in a single API.
> Application has to handle this.
> E.g.: One way by doing get() on position2 first, if not found then on
> position1 (it can use functions).
> There could be a window where the position2 is getting updated but its not
> yet reflected in the region. Depending on use-case requirement (if its fine
> to get old data when its getting updated), application has to handle it.
>
> -Anil.
>
>
> On Tue, Sep 25, 2018 at 10:40 PM aashish choudhary <
> aashish.choudhary1@gmail.com> wrote:
>
>> Use case description:-
>>
>> We have requirement where in we will have two geode regions let's say
>> position1 and position2 now position1 will have positions as of 25
>> september after end of the day a data load operation will run which will
>> put data to position2 region. At this point while data load is running any
>> get request will be served from position1 region because there is a
>> possibility of getting stale data. Now the client will have to make a
>> switch to position2 region to serve get request with latest data after the
>> load gets completed. So is there any way by which a client application
>> could make a switch to different region once the data load gets completed .
>>
>>
>> With best regards,
>> Ashish
>>
>

Re: Get geode region insert completion event

[Anilkumar Gingade <ag...@pivotal.io>]

Currently, there is no option to specify multiple region in a single API.
Application has to handle this.
E.g.: One way by doing get() on position2 first, if not found then on
position1 (it can use functions).
There could be a window where the position2 is getting updated but its not
yet reflected in the region. Depending on use-case requirement (if its fine
to get old data when its getting updated), application has to handle it.

-Anil.


On Tue, Sep 25, 2018 at 10:40 PM aashish choudhary <
aashish.choudhary1@gmail.com> wrote:

> Use case description:-
>
> We have requirement where in we will have two geode regions let's say
> position1 and position2 now position1 will have positions as of 25
> september after end of the day a data load operation will run which will
> put data to position2 region. At this point while data load is running any
> get request will be served from position1 region because there is a
> possibility of getting stale data. Now the client will have to make a
> switch to position2 region to serve get request with latest data after the
> load gets completed. So is there any way by which a client application
> could make a switch to different region once the data load gets completed .
>
>
> With best regards,
> Ashish
>