You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Jose Otavio Rizzatti <jo...@uber.com> on 2018/06/15 10:09:18 UTC
Off-heap eviction configuration
Hi everyone,
I'm having issues getting the off-heap eviction to work. The nodes in my
cluster are failing due to OutOfMemory errors.
I'm currently trying to get it to work like this:
IgniteConfuration igniteConfiguration = new IgniteConfiguration();
DataStorageConfiguration dataStorageConfig = new
DataStorageConfiguration();
DataRegionConfiguration dataRegionConfig = new
DataRegionConfiguration();
dataRegionConfig.setInitialSize((long) Math.ceil(0.2 *
offHeapMemoryMax)); // 20% of 256MB
dataRegionConfig.setMaxSize(offHeapMemoryMax); // 256MB, for testing
purposes
dataRegionConfig.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
dataRegionConfig.setEvictionThreshold(0.9);
dataRegionConfig.setName("OffHeapRegion");
// tried both default data region, and setting a data region list, but
neither
dataStorageConfig.setDefaultDataRegionConfiguration(dataRegionConfig);
igniteConfiguration.setDataStorageConfiguration(dataStorageConfig);
CacheConfiguration cacheConfiguration = new CacheConfiguration();
cacheConfiguration.setName("myCache");
cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
cacheConfiguration.setDataRegionName("OffHeapRegion");
cacheConfiguration.setBackups(1);
igniteConfiguration.setCacheConfiguration(cacheConfiguration);
Ignite.start(igniteConfiguration);
ignite.getOrCreateCache("myCache");
I can use the cache from my application, but once the current off-heap size
is reached, I see this in the logs:
ERROR [2018-06-15 09:37:16,095]
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache:
<metadata> Unexpected exception during cache update
! org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Not enough
memory allocated (consider increasing data region size or enabling
evictions) [policyName=OffHeapRegion, size=262.1 MB]
! at
org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl.allocatePage(PageMemoryNoStoreImpl.java:292)
! at
org.apache.ignite.internal.processors.cache.persistence.freelist.FreeListImpl.allocateDataPage(FreeListImpl.java:456)
! at
org.apache.ignite.internal.processors.cache.persistence.freelist.FreeListImpl.insertDataRow(FreeListImpl.java:494)
! at
org.apache.ignite.internal.processors.cache.persistence.RowStore.addRow(RowStore.java:90)
! at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.createRow(IgniteCacheOffheapManagerImpl.java:1255)
! at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:4408)
! at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:4204)
! at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:3918)
! at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:2988)
! at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$6200(BPlusTree.java:2882)
! at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1713)
! at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1602)
! ... 82 common frames omitted
The log mentions eviction being off and when checking the cache configs
with visorcmd, it says that eviction policy is off too.
What am I doing wrong here?
Thanks,
Jose Rizzatti
Re: Off-heap eviction configuration
Posted by dkarachentsev <dk...@gridgain.com>.
Check your configuration. This code works perfectly well for me. If set page
eviction mode to disabled - IOOME will be thrown:
IgniteConfiguration igniteConfiguration = new IgniteConfiguration();
DataStorageConfiguration dataStorageConfig = new
DataStorageConfiguration();
long offHeapMemoryMax = 256 * 1024 * 1024;
DataRegionConfiguration dataRegionConfig = new
DataRegionConfiguration();
dataRegionConfig.setInitialSize((long) Math.ceil(0.2 *
offHeapMemoryMax)); // 20% of 256MB
dataRegionConfig.setMaxSize(offHeapMemoryMax); // 256MB, for testing
purposes
dataRegionConfig.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
dataRegionConfig.setEvictionThreshold(0.9);
dataRegionConfig.setName("OffHeapRegion");
// tried both default data region, and setting a data region list,
but neither
dataStorageConfig.setDataRegionConfigurations(dataRegionConfig);
igniteConfiguration.setDataStorageConfiguration(dataStorageConfig);
CacheConfiguration cacheConfiguration = new CacheConfiguration();
cacheConfiguration.setName("myCache");
cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
cacheConfiguration.setDataRegionName("OffHeapRegion");
cacheConfiguration.setBackups(1);
igniteConfiguration.setCacheConfiguration(cacheConfiguration);
Ignite ignite = Ignition.start(igniteConfiguration);
IgniteCache<Object, Object> cache =
ignite.getOrCreateCache("myCache");
for (long i = Long.MIN_VALUE; i < Long.MAX_VALUE; i++)
cache.put(i, new byte[1800]);
Thanks!
-Dmitry
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Off-heap eviction configuration
Posted by Jose Otavio Rizzatti <jo...@uber.com>.
Hi Dmitry,
I tried that already but it didn't help either.
*Jose Otavio Rizzatti*
Senior Software Engineer - Mobile Developer Platform
+31 6 22257080 | joseotav@uber.com | uber.com
On Fri, Jun 15, 2018 at 1:48 PM dkarachentsev <dk...@gridgain.com>
wrote:
> Hi,
>
> I see you used your data region as default and set name for it. Try to set
> it to DataStorageConfiguration.setDataRegionConfigurations().
>
> Thanks!
> -Dmitry
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Off-heap eviction configuration
Posted by dkarachentsev <dk...@gridgain.com>.
Hi,
I see you used your data region as default and set name for it. Try to set
it to DataStorageConfiguration.setDataRegionConfigurations().
Thanks!
-Dmitry
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Off-heap eviction configuration
Posted by Jose Otavio Rizzatti <jo...@uber.com>.
Forgot to mention that I'm using Ignite 2.3. And the same behavior persists
if I try using the MemoryConfiguration APIs.
On Fri, Jun 15, 2018 at 12:09 PM Jose Otavio Rizzatti <jo...@uber.com>
wrote:
> Hi everyone,
>
> I'm having issues getting the off-heap eviction to work. The nodes in my
> cluster are failing due to OutOfMemory errors.
> I'm currently trying to get it to work like this:
>
> IgniteConfuration igniteConfiguration = new IgniteConfiguration();
>
> DataStorageConfiguration dataStorageConfig = new
> DataStorageConfiguration();
>
> DataRegionConfiguration dataRegionConfig = new
> DataRegionConfiguration();
> dataRegionConfig.setInitialSize((long) Math.ceil(0.2 *
> offHeapMemoryMax)); // 20% of 256MB
> dataRegionConfig.setMaxSize(offHeapMemoryMax); // 256MB, for testing
> purposes
>
> dataRegionConfig.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
> dataRegionConfig.setEvictionThreshold(0.9);
> dataRegionConfig.setName("OffHeapRegion");
>
> // tried both default data region, and setting a data region list, but
> neither
> dataStorageConfig.setDefaultDataRegionConfiguration(dataRegionConfig);
> igniteConfiguration.setDataStorageConfiguration(dataStorageConfig);
>
> CacheConfiguration cacheConfiguration = new CacheConfiguration();
> cacheConfiguration.setName("myCache");
> cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
> cacheConfiguration.setDataRegionName("OffHeapRegion");
> cacheConfiguration.setBackups(1);
> igniteConfiguration.setCacheConfiguration(cacheConfiguration);
>
> Ignite.start(igniteConfiguration);
> ignite.getOrCreateCache("myCache");
>
> I can use the cache from my application, but once the current off-heap
> size is reached, I see this in the logs:
>
> ERROR [2018-06-15 09:37:16,095]
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache:
> <metadata> Unexpected exception during cache update
> ! org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Not enough
> memory allocated (consider increasing data region size or enabling
> evictions) [policyName=OffHeapRegion, size=262.1 MB]
> ! at
> org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl.allocatePage(PageMemoryNoStoreImpl.java:292)
> ! at
> org.apache.ignite.internal.processors.cache.persistence.freelist.FreeListImpl.allocateDataPage(FreeListImpl.java:456)
> ! at
> org.apache.ignite.internal.processors.cache.persistence.freelist.FreeListImpl.insertDataRow(FreeListImpl.java:494)
> ! at
> org.apache.ignite.internal.processors.cache.persistence.RowStore.addRow(RowStore.java:90)
> ! at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.createRow(IgniteCacheOffheapManagerImpl.java:1255)
> ! at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:4408)
> ! at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:4204)
> ! at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:3918)
> ! at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:2988)
> ! at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$6200(BPlusTree.java:2882)
> ! at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1713)
> ! at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1602)
> ! ... 82 common frames omitted
>
> The log mentions eviction being off and when checking the cache configs
> with visorcmd, it says that eviction policy is off too.
> What am I doing wrong here?
>
> Thanks,
>
> Jose Rizzatti
>