You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by SidP <si...@msci.com> on 2020/01/09 11:46:19 UTC
After increasing maxsize in DataRegionConfiguration...Ignite throws
out of memory exception
I am trying to start Ignite as follows
I am running ignite on windows 10 and i have 32 GB RAM out of which 22 GB is
free available.
If i specify maxsize = 1gb it works fine but for maxsize=2gb it fails., why?
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="systemWorkerBlockedTimeout" value="#{5 * 60 * 1000}"/>
<property name="failureDetectionTimeout" value="120000"/>
<property name="clientFailureDetectionTimeout" value="120000"/>
<property name="failureHandler">
<bean class="org.apache.ignite.failure.StopNodeFailureHandler">
<property name="ignoredFailureTypes">
<list>
</list>
</property>
</bean>
</property>
<property name="cacheConfiguration">
<list>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="WorkbenchCache"/>
<property name="atomicityMode" value="ATOMIC"/>
<property name="backups" value="1"/>
</bean>
</list>
</property>
<property name="dataStorageConfiguration">
<bean class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="persistenceEnabled" value="true" />
<property name="name" value="Default_Region"/>
<property name="initialSize" value="#{500L * 1024 * 1024}"/>
*<property name="maxSize" value="#{1L * 1024 * 1024 * 1024}"/>*
</bean>
</property>
<property name="walMode" value="FSYNC"/>
<property name="writeThrottlingEnabled" value="true" />
<property name="pageSize" value="4096"/>
</bean>
</property>
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>MyHostName1:47500..47501</value>
<value>MyHostName2:47500..47501</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
<property name="communicationSpi">
<bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
<property name="localPort" value="48100" />
<property name="localPortRange" value="10" />
<property name="usePairedConnections" value="true"></property>
<property name="messageQueueLimit" value="1024"></property>
<property name="directSendBuffer" value="true"></property>
<property name="selectorsCount" value="4"></property>
<property name="socketWriteTimeout" value="300000" />
</bean>
</property>
</bean>
</beans>
Now as soon i increase <property name="maxSize" value="#{2L * 1024 * 1024 *
1024}"/> 2gb onward it throws out of memory exception
class Program
{
public static IIgnite IgniteServer;
static void Main(string[] args)
{
try
{
String EnvironmentPath =
Environment.GetEnvironmentVariable("IGNITE_HOME");
Environment.SetEnvironmentVariable("_JAVA_OPTIONS",
"-Xms1g");
var _configPath = Path.Combine(EnvironmentPath,
"config\\ignite_server_config.xml");
if (File.Exists(_configPath))
{
Console.WriteLine(_configPath);
IgniteServer = Ignition.Start(_configPath);
Console.WriteLine("server started");
//Activate cluster
IgniteServer.GetCluster().SetActive(true);
Console.WriteLine("Cluster activated");
//Get all server nodes which are up and running.
ICollection<IClusterNode> nodes =
IgniteServer.GetCluster().ForServers().GetNodes();
//Set baseline topology that is represented by these
nodes
IgniteServer.GetCluster().SetBaselineTopology(nodes);
}
}
catch(Exception ex)
{
Console.WriteLine(ex);
}
}
}
Ignite Log:
[11:40:37,161][INFO][exchange-worker-#43][FsyncModeFileWriteAheadLogManager]
Resolved write ahead log archive directory:
E:\apache-ignite-2.7.6-bin\work\db\wal\archive\node00-8a0f96ea-f6d6-42dd-b9bc-7a61d88fbf88
[11:40:37,161][INFO][exchange-worker-#43][FsyncModeFileWriteAheadLogManager]
Started write-ahead log manager [mode=FSYNC]
[11:40:37,161][INFO][main][IgniteKernal] Performance suggestions for grid
(fix if possible)
[11:40:37,161][INFO][main][IgniteKernal] To disable, set
-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
[11:40:37,161][INFO][main][IgniteKernal] ^-- Enable server mode for JVM
(add '-server' to JVM options)
[11:40:37,161][INFO][main][IgniteKernal] ^-- Enable G1 Garbage Collector
(add '-XX:+UseG1GC' to JVM options)
[11:40:37,161][INFO][main][IgniteKernal] ^-- Specify JVM heap max size
(add '-Xmx<size>[g|G|m|M|k|K]' to JVM options)
[11:40:37,161][INFO][main][IgniteKernal] ^-- Set max direct memory size if
getting 'OOME: Direct buffer memory' (add
'-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options)
[11:40:37,161][INFO][main][IgniteKernal] ^-- Disable processing of calls
to System.gc() (add '-XX:+DisableExplicitGC' to JVM options)
[11:40:37,161][INFO][main][IgniteKernal] Refer to this page for more
performance suggestions:
https://apacheignite.readme.io/docs/jvm-and-system-tuning
[11:40:37,161][INFO][main][IgniteKernal]
[11:40:37,161][INFO][exchange-worker-#43][PageMemoryImpl] Started page
memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB,
checkpointBuffer=100.0 MiB]
[11:40:37,161][INFO][main][IgniteKernal] To start Console Management &
Monitoring run ignitevisorcmd.{sh|bat}
[11:40:37,177][INFO][exchange-worker-#43][PageMemoryImpl] Started page
memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB,
checkpointBuffer=100.0 MiB]
[11:40:37,177][INFO][main][IgniteKernal] Data Regions Configured:
[11:40:37,177][INFO][main][IgniteKernal] ^-- Default_Region
[initSize=500.0 MiB, maxSize=2.0 GiB, persistence=true]
[11:40:37,177][INFO][main][IgniteKernal]
>>> +----------------------------------------------------------------------+
>>> Ignite ver. 2.7.6#20190911-sha1:21f7ca41c4348909e2fd26ccf59b5b2ce1f4474e
>>> +----------------------------------------------------------------------+
>>> OS name: Windows Server 2008 R2 6.1 x86
>>> CPU(s): 8
>>> Heap: 1.0GB
>>> VM name: 5616@******
>>> Local node [ID=4045A130-7CB5-49EB-B483-BC8B7CF0BDA1, order=1,
>>> clientMode=false]
>>> Local node addresses: [********]
>>> Local ports: TCP:10800 TCP:11211 TCP:47500 TCP:48100
[11:40:37,177][SEVERE][exchange-worker-#43][GridDhtPartitionsExchangeFuture]
Failed to reinitialize local partitions (rebalancing will be stopped):
GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1,
minorTopVer=1], discoEvt=DiscoveryCustomEvent
[customMsg=ChangeGlobalStateMessage
[id=d502b1a8f61-5791bb11-5e5b-4b85-b04c-e1d1f9bbfd6a,
reqId=f9a4d984-f34f-492a-ba10-bbd2d1d0318b,
initiatingNodeId=4045a130-7cb5-49eb-b483-bc8b7cf0bda1, activate=true,
baselineTopology=BaselineTopology [id=0, branchingHash=501879262,
branchingType='Cluster activation',
baselineNodes=[8a0f96ea-f6d6-42dd-b9bc-7a61d88fbf88]],
forceChangeBaselineTopology=false, timestamp=1578570037052],
affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=4045a130-7cb5-49eb-b483-bc8b7cf0bda1, addrs=[0:0:0:0:0:0:0:1,
10.62.21.53, 10.62.44.30, 10.63.216.30, 127.0.0.1],
sockAddrs=[UEU-IP-WAPP0002.coresit.msci.org/10.63.216.30:47500,
/10.62.44.30:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
/10.62.21.53:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1578570033978, loc=true, ver=2.7.6#20190911-sha1:21f7ca41,
isClient=false], topVer=1, nodeId8=4045a130, msg=null,
type=DISCOVERY_CUSTOM_EVT, tstamp=1578570037145]], nodeId=4045a130,
evt=DISCOVERY_CUSTOM_EVT]
java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at
org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1205)
at
org.apache.ignite.internal.mem.unsafe.UnsafeMemoryProvider.nextRegion(UnsafeMemoryProvider.java:103)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$6.nextRegion(GridCacheDatabaseSharedManager.java:1131)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.start(PageMemoryImpl.java:356)
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.startMemoryPolicies(IgniteCacheDatabaseSharedManager.java:224)
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.onActivate(IgniteCacheDatabaseSharedManager.java:1148)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.onActivate(GridCacheDatabaseSharedManager.java:709)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedContext.activate(GridCacheSharedContext.java:278)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:1082)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:736)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2681)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2553)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Unknown Source)
[11:40:37,177][INFO][main][GridDiscoveryManager] Topology snapshot [ver=1,
locNode=4045a130, servers=1, clients=0, state=INACTIVE, CPUs=8,
offheap=2.0GB, heap=1.0GB]
[11:40:37,177][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture]
Finish exchange future [startVer=AffinityTopologyVersion [topVer=1,
minorTopVer=1], resVer=null, err=java.lang.OutOfMemoryError]
[11:40:37,177][INFO][main][GridDiscoveryManager] ^-- Baseline [id=0,
size=1, online=1, offline=0]
[11:40:37,177][INFO][main][GridDiscoveryManager] ^-- All baseline nodes
are online, will start auto-activation
[11:40:37,177][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture]
Completed partition exchange
[localNode=4045a130-7cb5-49eb-b483-bc8b7cf0bda1,
exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion
[topVer=1, minorTopVer=1], evt=DISCOVERY_CUSTOM_EVT,
evtNode=TcpDiscoveryNode [id=4045a130-7cb5-49eb-b483-bc8b7cf0bda1,
addrs=[0:0:0:0:0:0:0:1, 10.62.21.53, 10.62.44.30, 10.63.216.30, 127.0.0.1],
sockAddrs=[UEU-IP-WAPP0002.coresit.msci.org/10.63.216.30:47500,
/10.62.44.30:47500, /0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
/10.62.21.53:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1578570033978, loc=true, ver=2.7.6#20190911-sha1:21f7ca41,
isClient=false], done=true], topVer=null, durationFromInit=32]
[11:40:37,177][SEVERE][exchange-worker-#43][] Critical system error
detected. Will be handled accordingly to configured handler
[hnd=StopNodeFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=[]]], failureCtx=FailureContext [type=CRITICAL_ERROR,
err=java.lang.OutOfMemoryError]]
java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at
org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1205)
at
org.apache.ignite.internal.mem.unsafe.UnsafeMemoryProvider.nextRegion(UnsafeMemoryProvider.java:103)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$6.nextRegion(GridCacheDatabaseSharedManager.java:1131)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.start(PageMemoryImpl.java:356)
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.startMemoryPolicies(IgniteCacheDatabaseSharedManager.java:224)
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.onActivate(IgniteCacheDatabaseSharedManager.java:1148)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.onActivate(GridCacheDatabaseSharedManager.java:709)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedContext.activate(GridCacheSharedContext.java:278)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:1082)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:736)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2681)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2553)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Unknown Source)
[11:40:37,192][WARNING][main][] GC server mode is not enabled, this could
lead to less than optimal performance on multi-core machines (to enable see
http://msdn.microsoft.com/en-us/library/ms229357(v=vs.110).aspx).
[11:40:37,192][INFO][main][PluginProcessor] Configured .NET plugins:
[11:40:37,192][WARNING][exchange-worker-#43][FailureProcessor] No deadlocked
threads detected.
[11:40:37,192][INFO][main][PluginProcessor] ^-- None
[11:40:37,192][WARNING][exchange-worker-#43][FailureProcessor] Thread dump
at 2020/01/09 11:40:37 UTC
Thread [name="wal-file-archiver%null-#44", id=73, state=WAITING, blockCnt=0,
waitCnt=1]
Lock
[object=o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileArchiver@1212090,
ownerName=null, ownerId=-1]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Unknown Source)
at
o.a.i.i.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager$FileArchiver.body(FsyncModeFileWriteAheadLogManager.java:1581)
at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Unknown Source)
Thread [name="session-timeout-worker-#34", id=50, state=TIMED_WAITING,
blockCnt=0, waitCnt=1]
at java.lang.Thread.sleep(Native Method)
at
o.a.i.i.processors.rest.GridRestProcessor$4.body(GridRestProcessor.java:495)
at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Unknown Source)
Thread [name="exchange-worker-#43", id=72, state=RUNNABLE, blockCnt=2,
waitCnt=1]
at sun.management.ThreadImpl.dumpThreads0(Native Method)
at sun.management.ThreadImpl.dumpAllThreads(Unknown Source)
at o.a.i.i.util.IgniteUtils.dumpThreads(IgniteUtils.java:1368)
at
o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:128)
- locked o.a.i.i.processors.failure.FailureProcessor@f8f726
at
o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:104)
at
o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2567)
at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Unknown Source)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: After increasing maxsize in DataRegionConfiguration...Ignite
throws out of memory exception
Posted by Tunas <si...@msci.com>.
I am not allocating large memory. I just want to allocate at least 2gb of
RAM from available 22GB free space.
I will try with 64-bit JVM.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: After increasing maxsize in DataRegionConfiguration...Ignite
throws out of memory exception
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Please ensure that you are using 64 bit JVM. You may not be able to
allocate large chunks of memory when using 32-bit VM:
>>> OS name: Windows Server 2008 R2 6.1 *x86*
Regards,
--
Ilya Kasnacheev
пт, 10 янв. 2020 г. в 08:45, Tunas <si...@msci.com>:
> Can someone please give me some input. I am stuck on this since last 2
> days.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: After increasing maxsize in DataRegionConfiguration...Ignite
throws out of memory exception
Posted by Tunas <si...@msci.com>.
Can someone please give me some input. I am stuck on this since last 2 days.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/