Cache startSize tunning

[diopek <de...@gmail.com>]

Is it possible to use cache config xml file and CacheConfig API together.

In my use cache, cache size will change per batch run as at the end of each
batch cache entries will be flushed and cache will be re-populated. Also
each time, finalSize=startSize  of the cache is known in advance. 
Is it possible to keep all the configuration defined in cache xml file as
default, just set the startSize using CacheConfiguration API.  As per
'Performance Tips' documentation, the process of internal cache map
re-sizing is CPU-intensive and time-consuming, I want to avoid that.
please advise.




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cache-startSize-tunning-tp1948.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cache startSize tunning

[Denis Magda <dm...@gridgain.com>]

Hi,

Please see below...

On 11/13/2015 7:07 AM, diopek wrote:
> Below is one use case;
> Step 1, I get a handle to cache already configured in one common cache
> configuration xml file
> Step 2, flush the cache in case it is full
> Step 3, populate regular Java LinkedHashMap from one DB
> Step 4, add that LinkedHashMap to Ignite cache.
> Before Step 4 I want to set the Ignite Cache size same as LinkedHashMap
> returned from DB.
There is no way to change the start size of already created Ignite Cache.
Each Ignite node, that stores partitions of an Ignite Cache, stores them 
in a specific hash map data structure.
Thus changing the size explicitly, after the map has already been 
created before, will require to perform rehashing in any case and this 
is almost the same as a recreation of the map from scratch.
So there is no sense to have such an operation in hash maps and in 
Ignite Cache as well (which in reality is a distributed hash map).

> [1] IgniteCache<Integer, ArrayList&lt;MyDTO>> cache = Ignition.ignite()			
> .getOrCreateCache(MY_CACHE);
> [2]		cache.clear();
> [3]		LinkedHashMap<Integer, ArrayList&lt;MyDTO>> res =
> dstreamDAO.getExistingMap(asOfDate, scenarioId);
> [4]		cache.putAll(res);
>
> One other question, this above is one typical use case that I populate
> Ignite Cache from existing APIs for regular Java Map, is there any
> inefficiency on this method, should I populate Ignite cache with directly
> rather than using putAll method passing LinkedHashMap. Thanks.
As an alternative to putAll you can use IgniteDataStreamer API that 
works perfectly well for the use cases of initial cache preloading.
https://apacheignite.readme.io/docs/data-streamers

In addition Ignite has CacheStore interface that is used in cases when 
it's needed to preload data from an underlying persistent storage like DB
https://apacheignite.readme.io/docs/persistent-store#cachestore

Regards,
Denis
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cache-startSize-tunning-tp1948p1955.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cache startSize tunning

[diopek <de...@gmail.com>]

Below is one use case;
Step 1, I get a handle to cache already configured in one common cache
configuration xml file
Step 2, flush the cache in case it is full
Step 3, populate regular Java LinkedHashMap from one DB
Step 4, add that LinkedHashMap to Ignite cache.
Before Step 4 I want to set the Ignite Cache size same as LinkedHashMap
returned from DB.

[1] IgniteCache<Integer, ArrayList&lt;MyDTO>> cache = Ignition.ignite()			
.getOrCreateCache(MY_CACHE);
[2]		cache.clear();
[3]		LinkedHashMap<Integer, ArrayList&lt;MyDTO>> res =
dstreamDAO.getExistingMap(asOfDate, scenarioId);
[4]		cache.putAll(res);

One other question, this above is one typical use case that I populate
Ignite Cache from existing APIs for regular Java Map, is there any
inefficiency on this method, should I populate Ignite cache with directly
rather than using putAll method passing LinkedHashMap. Thanks.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cache-startSize-tunning-tp1948p1955.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cache startSize tunning

[vkulichenko <va...@gmail.com>]

Hi,


diopek wrote
> Is it possible to keep all the configuration defined in cache xml file as
> default, just set the startSize using CacheConfiguration API.

You can put your cache configuration template outside IgniteConfiguration
bean and then use Ignition.loadSpringBean() method.

For example, if you configuration look like this:

<beans>
    <bean id="my-cache-template"
class="org.apache.ignite.configuration.CacheConfiguration">
        ...
    </bean>

    <bean class="org.apache.ignite.configuration.IgniteConfiguration">
        ...
    </bean>
</beans>

You can start a cache with the modified start size like this:

CacheConfiguration cacheCfg = Ignition.loadSpringBean("path/to/config.xml",
"my-cache-template");
cacheCfg.setName("cache-with-start-size-1000");
cacheCfg.setStartSize(1000);
IgniteCache cache = ignite.startCache(cacheCfg);

Let us know if it works for you.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cache-startSize-tunning-tp1948p1953.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.