Questions about "ignite-indexing"

[Chris Berry <ch...@gmail.com>]

Hi,

I have a few questions about efficiency and the “ignite-indexing” module…
(Sorry, I asked this inside another thread, but wanted to surface it as
first-class questions)

I have the following code. Which is clearly inefficient.

    public Set<TKey> getCacheKeys() { 
        String tableName = valueClass.getSimpleName(); 
        SqlFieldsQuery qry = new SqlFieldsQuery("select _key from " +
tableName); 
        Collection<List&lt;?>> res =
 igniteCacheHandle.query(qry).getAll(); 

        Set<TKey> keys = new HashSet<>(); 
        for(List<?> row : res) { 
            keys.add((TKey)row.get(0)); 
        } 
        return keys; 
    } 

Using a SqlFieldsQuery. 

And I use it like this: 

    public List<Foo> findMatches(SearchTerms searchTerms) { 
        Map<String, Foot> map = cache.getAll(cache.getCacheKeys()); 
        return matchingStrategy.match(map, searchTerms); 
    } 

I wanted to get something working – which it is.
But it is clearly inefficient (get all the Keys, to get all the Entries, to
create a local Map)
And my perf-tests validate that this is not the best way to accomplish this.
(Even though this is a relatively small cache) 

So. I have a few questions. 

1) I have only one REPLICATED cache – out of many caches -- that I need to
do a Query against. 
The rest are all used as simple KV stores. 

Have I introduced an inefficiency by introducing the ignite-indexing
module?? 
Is there a way to use it ONLY for 1 cache?? 

Or is indexing always present, and I have just now become aware of it?? 
(This seems unlikely since the H2 DB issue I encountered when I introduced
“ignite-indexing” should have bitten me sooner??) But, of course, some form
of index must be there to allow the K/V lookup...

2) Would it be more efficient to use a ScanQuery above?? 
Or is the SqlFieldsQuery roughly equivalent?? 

NOTE: I assume to do this whole thing more efficiently I would
use a ScanQuery with a Filter?? 
Which would eliminate the need for the interim copies. 
Is that correct??

3) I am using:  igniteConfig.setMarshaller(new BinaryMarshaller())  --
paired with:  cache.withKeepBinary() 
Do I need to do anything special in this case?? 
Presently it seems not?? 

NOTE:  I am using only this: igniteCacheConfig.setIndexedTypes(keyClass,
valueClass); 
And it all is working as expected. 
Still. If I used IgniteBiPredictae would it be more efficient?? 

Thanks,
-- Chris  




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Questions-about-ignite-indexing-tp16292.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Questions about "ignite-indexing"

["slava.koptilin" <sl...@gmail.com>]

Hi Chris,

It looks like Dmitry already answered your qns:
http://apache-ignite-users.70518.x6.nabble.com/getAsMap-tt16242.html#a16347


> Hi Chris, 
> 
> 1. Indexing may slowdown insertions, but in will not be used if no
> CacheConfiguration.setIndexedTypes() or
> CacheConfiguration.setQueryEntities() set. 
> 
> 2. It depends on conditions. If you have a lot of data, but you need to
> filter out a small set of them, then indexing may greatly help you,
> because it will use index but full scan. In other hand, scan query may be
> greatly optimized if it's used locally and per partition, but again, it
> will check all entries. 
> 
> 3. If you set withKeepBinary(), then you'll be working with BinaryObject
> and it's not require any specific config. 
> 
> Thanks! 
> -Dmitry.





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Questions-about-ignite-indexing-tp16292p16369.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.