You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by nithin91 <ni...@franklintempleton.com> on 2020/03/17 14:46:58 UTC
Data Isolation during Ignite Cache load
Hi ,
We have Created a cache in Ignite that gets loaded with Data from
Oracle.Cache is getting loaded using JDBC POJO Store method
Ignite.Cache("Cache Name").loadCache(null).Can anyone explain how will
ignite handle the below scenario when loading data using the method
Ignite.Cache("Cache Name").loadCache(null).
*Scenario:*
We have built a rest service on top this cache and sends the data in the
cache as response after applying some filters.Now During the time of Data
load, if the user sends a get request and when we try to fetch the data from
ignite cache will the filters be applied only the data that is loaded so far
or does the filters be applied on the data that is available prior to the
start of data load.
*Please note we have set the Atomicity mode is set as Atomic for cache.*
In case if the filters are applied only on the data that is loaded so far,
then is there a property that i need to set so that until the cache load is
loaded , any query on the cache should fetch only the data that is available
prior to the start of data load.Currently as per my observation, i am able
to see that the filters are applied only on the data that is loaded so far.
Also can anyone help me the possible ways to schedule the Ignite cache
refresh such that the ignite cache refresh job has to be triggered on the
success condition of another unix shell script job.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Data Isolation during Ignite Cache load
Posted by Evgenii Zhuravlev <e....@gmail.com>.
Hi,
> ignite should display the data prior to data load instead of displaying
the data that isloaded till that point
*What do you mean? How it will help with a problem that you describing
after that? *
> he might get an empty response in case if the requested data is notyet
loaded when the cache load process is running
There is no data in Ignite, that's why it will not return any value. I
don't really understand, why in this case UI will allow user to query entry
which was not loaded yet.
Evgenii
пн, 23 мар. 2020 г. в 08:45, nithin91 <
nithinbharadwaj.govindaraju@franklintempleton.com>:
> Hi ,
>
> But it is not behaving that way. I have tested it using the following
> scenario.
>
> Scenario:
>
>
> Initially loading 1000 rows from oracle table with the following filter
> where rownum<=1000 into ignite cache.
> During the cache load process, i checked the count using sql fields query i
> got the response as number of records that are loaded into the cache till
> that point i.e 100,200 and finally 1000.
>
> Again started the load into ignite cache from oracle table with the same
> filter where rownum<=1000.
> Now during the cache load time, till 1000 rows are loaded it is showing the
> count as 1000 but once the count crosses 1000 it is showing the count as
> 1001,1002 and finally 2000.
>
> Can you please help me fix this issue (i.e While loading Ignite Cache using
> JDBC POJO Ignite.cache.loadcache method, ignite should display the data
> prior to the start of data load instead of displaying the data that is
> loaded till that point when a query is submitted during the load process.
> As
> we are using the ignite cache for front-end websites,there might be
> scenarios where the user queries the cache against a specific value for
> which he might get an empty response in case if the requested data is not
> yet loaded when the cache load process is running. Instead the problem will
> be resolved if ignite always shows the data prior to start of data load if
> the cache is queried during data load process).
>
>
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Data Isolation during Ignite Cache load
Posted by nithin91 <ni...@franklintempleton.com>.
Hi ,
But it is not behaving that way. I have tested it using the following
scenario.
Scenario:
Initially loading 1000 rows from oracle table with the following filter
where rownum<=1000 into ignite cache.
During the cache load process, i checked the count using sql fields query i
got the response as number of records that are loaded into the cache till
that point i.e 100,200 and finally 1000.
Again started the load into ignite cache from oracle table with the same
filter where rownum<=1000.
Now during the cache load time, till 1000 rows are loaded it is showing the
count as 1000 but once the count crosses 1000 it is showing the count as
1001,1002 and finally 2000.
Can you please help me fix this issue (i.e While loading Ignite Cache using
JDBC POJO Ignite.cache.loadcache method, ignite should display the data
prior to the start of data load instead of displaying the data that is
loaded till that point when a query is submitted during the load process. As
we are using the ignite cache for front-end websites,there might be
scenarios where the user queries the cache against a specific value for
which he might get an empty response in case if the requested data is not
yet loaded when the cache load process is running. Instead the problem will
be resolved if ignite always shows the data prior to start of data load if
the cache is queried during data load process).
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Data Isolation during Ignite Cache load
Posted by Evgenii Zhuravlev <e....@gmail.com>.
Yes, SQL queries use data that was already preloaded only.
Evgenii
ср, 18 мар. 2020 г. в 01:23, nithin91 <
nithinbharadwaj.govindaraju@franklintempleton.com>:
> Hi ,
>
> Filter in the sense, SQL Fields Query with where clause is executed. I
> hope
> the following example helps you understand my query.
>
> *Scenario:*
>
> For example there is a cache named Products which has 1 million records
> with
> two distinct product categories(i.e. Furniture,Electronics) among these 1
> million records.
>
> Now once the Data load is started , if i execute a sql fields query like
> below, will i get the count as 1 million or only the count of records that
> are loaded so far.
>
> select count(1) from Products where category='Electronics'
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Data Isolation during Ignite Cache load
Posted by nithin91 <ni...@franklintempleton.com>.
Hi ,
Filter in the sense, SQL Fields Query with where clause is executed. I hope
the following example helps you understand my query.
*Scenario:*
For example there is a cache named Products which has 1 million records with
two distinct product categories(i.e. Furniture,Electronics) among these 1
million records.
Now once the Data load is started , if i execute a sql fields query like
below, will i get the count as 1 million or only the count of records that
are loaded so far.
select count(1) from Products where category='Electronics'
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Data Isolation during Ignite Cache load
Posted by Evgenii Zhuravlev <e....@gmail.com>.
Hi,
> will the filters be applied
What kind of filters? Do you mean SQL queries?
Evgenii
вт, 17 мар. 2020 г. в 07:47, nithin91 <
nithinbharadwaj.govindaraju@franklintempleton.com>:
> Hi ,
>
> We have Created a cache in Ignite that gets loaded with Data from
> Oracle.Cache is getting loaded using JDBC POJO Store method
> Ignite.Cache("Cache Name").loadCache(null).Can anyone explain how will
> ignite handle the below scenario when loading data using the method
> Ignite.Cache("Cache Name").loadCache(null).
>
> *Scenario:*
>
> We have built a rest service on top this cache and sends the data in the
> cache as response after applying some filters.Now During the time of Data
> load, if the user sends a get request and when we try to fetch the data
> from
> ignite cache will the filters be applied only the data that is loaded so
> far
> or does the filters be applied on the data that is available prior to the
> start of data load.
> *Please note we have set the Atomicity mode is set as Atomic for cache.*
>
>
> In case if the filters are applied only on the data that is loaded so far,
> then is there a property that i need to set so that until the cache load is
> loaded , any query on the cache should fetch only the data that is
> available
> prior to the start of data load.Currently as per my observation, i am able
> to see that the filters are applied only on the data that is loaded so far.
>
> Also can anyone help me the possible ways to schedule the Ignite cache
> refresh such that the ignite cache refresh job has to be triggered on the
> success condition of another unix shell script job.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>