You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by diopek <de...@gmail.com> on 2017/02/18 22:30:17 UTC
Ignite cache range query using cache keys
What is the possible query API to do range query using cache key
For example, for the following cache
IgniteCache<Long, MyPOJO>, where key is Java Long and value is POJO,
So, how can we retrieve all the Objects in a single API call using criteria
like;
*' key >= 1 AND key <= 5000'*
SqlQuery API provides capability to create SQL string by referring cache
value object fields by their Java attribute names in SQL string. So, is
there any default symbol like 'key' for Cache keys so that can be used in a
SQL String as above. Thanks,
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by diopek <de...@gmail.com>.
Basically, we just need indexing on keys, as we only query via keys (which is
Long type).
Value is is ArrayList<MyPOJO> which we don't need to query.
Is there any way to add index only on keys.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p10786.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by vkulichenko <va...@gmail.com>.
You need to provide correct value class when executing a query, in your case
it should be ArrayList. The "ArrayList<MyPOJO>.class" that you provided
there doesn't make any sense.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p13085.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by fatality <fa...@gmail.com>.
Hi
Could you remove cacheCfg.setIndexedTypes(Long.class,ArrayList.class) and
only leave cacheCfg.setIndexedTypes(Long.class,YourPojo.class);
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p13042.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by diopek <de...@gmail.com>.
we are doing pagination (like processing 5000 records at a time) during batch
processing. thats where we use range query using cache keys, ScanWuery
worked fine, SqlQuery throws the following exception that I posted earlier.
we were debugging some missing records issue, I wanted to change cache query
methodology from ScanQuery to SqlQuery to see if it makes any difference. In
our use case acccess to Ignite cache is local.
caused by: javax.cache.CacheException: Failed to find SQL table for type:
ArrayList<MyPOJO>.class
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p10824.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by vkulichenko <va...@gmail.com>.
Hi,
If you search only based on key, I would recommend to use get() and getAll()
methods instead of SQL.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p10815.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by diopek <de...@gmail.com>.
While SqlQuery fails, ScanQuery works without any issue.
Another issue we've notices some records are being missed when data loaded
into IgniteCache gets larger.
cache size() method returns 522511 records, 64 records less that 522575
which should be actual number of records in Ignite cache.
if we replace Ignite cache with regular Java LinkedHashMap records matches.
I posted the this issue also on separate thread.
Thanks in advance for your help.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p10812.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by diopek <de...@gmail.com>.
Hi Denis
After adding cacheConfig.setIndexedTypes(Long.class, ArrayList.class), below
is my current configuration;
*CacheConfiguration<Long, ArrayList<MyPOJO>> cacheCfg = new
CacheConfiguration<>(cacheName);
cacheCfg.setStartSize(startSize);
cacheCfg.setCacheMode(CacheMode.LOCAL);
cacheCfg.setIndexedTypes(Long.class,ArrayList.class);
cacheCfg.setCacheStoreFactory(FactoryBuilder.factoryOf(MyCacheLoadOnlyStore.class))*
After running the query that I mentioned earlier, I got the following
exception,
*Caused by: javax.cache.CacheException: Failed to find SQL table for type:
ArrayList<MyPOJO>.class
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$6.applyx(GridQueryProcessor.java:879)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$6.applyx(GridQueryProcessor.java:866)*
Not sure, what this exception means exactly. BTW, we are using Ignite 1.8.0
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p10805.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by Denis Magda <dm...@apache.org>.
> On Feb 21, 2017, at 11:48 PM, diopek <de...@gmail.com> wrote:
>
> Hi Denis,
> we have ignite cache that has key value pair as the following;
> IgniteCache<Long, ArrayList<MyPOJO>>
> and cache cofiguration for this cache is as the following;
> CacheConfiguration<Long, ArrayList<MyPOJO>> cacheCfg = new
> CacheConfiguration<>(cacheName);
> cacheCfg.setStartSize(startSize);
> cacheCfg.setCacheMode(CacheMode.LOCAL);
>
> I wrote query as the following;
> String sql = "_key >= ? && _key < ? ";
> IgniteCache<Long, ArrayList<MyPOJO>> igniteCache =
> Ignition.ignite().cache(this.getCacheName());
> SqlQuery<Long, ArrayList<MyPOJO>> sqlQuery = new SqlQuery<Long,
> ArrayList<MyPOJO>>("ArrayList<MyPOJO>.class", sql);
> sqlQuery.setArgs(min, max);
> igniteCache.query(sqlQuery).getAll();
>
Is there any significant reason why you store a list of MyPojos in a cache rather than individual objects? Just want to make a precaution that if some day you decide to query using MyPojo's fields than you’ll not be able to leverage from indexes because the objects will be inside of the lists.
> above method call returns the following exception;
> Caused by: javax.cache.CacheException: Indexing is disabled for cache:
> MY_CACHE. Use setIndexedTypes or setTypeMetadata methods on
> CacheConfiguration to enable.
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxy.validate(IgniteCacheProxy.java:852)
>
> I think I need to add indexing by calling
> cacheConfig.setIndexedTypes(Long.class, Value.class)
Do it this way - cacheConfig.setIndexedTypes(Long.class, ArrayList.class).
—
Denis
> But my value class is ArrayList<MyPOJO> not sure how to specify *.class file
> for this generic collection type. Can you please advise. Thanks,
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p10785.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by diopek <de...@gmail.com>.
Hi Denis,
we have ignite cache that has key value pair as the following;
IgniteCache<Long, ArrayList<MyPOJO>>
and cache cofiguration for this cache is as the following;
CacheConfiguration<Long, ArrayList<MyPOJO>> cacheCfg = new
CacheConfiguration<>(cacheName);
cacheCfg.setStartSize(startSize);
cacheCfg.setCacheMode(CacheMode.LOCAL);
I wrote query as the following;
String sql = "_key >= ? && _key < ? ";
IgniteCache<Long, ArrayList<MyPOJO>> igniteCache =
Ignition.ignite().cache(this.getCacheName());
SqlQuery<Long, ArrayList<MyPOJO>> sqlQuery = new SqlQuery<Long,
ArrayList<MyPOJO>>("ArrayList<MyPOJO>.class", sql);
sqlQuery.setArgs(min, max);
igniteCache.query(sqlQuery).getAll();
above method call returns the following exception;
Caused by: javax.cache.CacheException: Indexing is disabled for cache:
MY_CACHE. Use setIndexedTypes or setTypeMetadata methods on
CacheConfiguration to enable.
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.validate(IgniteCacheProxy.java:852)
I think I need to add indexing by calling
cacheConfig.setIndexedTypes(Long.class, Value.class)
But my value class is ArrayList<MyPOJO> not sure how to specify *.class file
for this generic collection type. Can you please advise. Thanks,
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717p10785.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite cache range query using cache keys
Posted by Denis Magda <dm...@apache.org>.
Hi,
Use “_key” to access a key in an SQL query.
Updated the bottom of this documentation section:
http://apacheignite.gridgain.org/v1.8/docs/sql-queries#query-types <http://apacheignite.gridgain.org/v1.8/docs/sql-queries#query-types>
"Use _key and _value keywords in an SQL query in order to compare to an entry's complete key or value rather than to individual fields. Apply the same keywords if you need to return a key or a value as a result of an SQL query execution.”
—
Denis
> On Feb 18, 2017, at 2:30 PM, diopek <de...@gmail.com> wrote:
>
> What is the possible query API to do range query using cache key
> For example, for the following cache
> IgniteCache<Long, MyPOJO>, where key is Java Long and value is POJO,
>
> So, how can we retrieve all the Objects in a single API call using criteria
> like;
>
> *' key >= 1 AND key <= 5000'*
>
> SqlQuery API provides capability to create SQL string by referring cache
> value object fields by their Java attribute names in SQL string. So, is
> there any default symbol like 'key' for Cache keys so that can be used in a
> SQL String as above. Thanks,
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-cache-range-query-using-cache-keys-tp10717.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.