You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by smovva <su...@sturfee.com> on 2018/06/30 20:43:45 UTC
Uneven partitioning and data not on off-heap memory.
I have started my cluster and pushed some data into a cache. Cluster details
are provided below. The cache has persistence enabled.
I see a couple of issues with how data is cached.
1. Out of 8 nodes only 2 nodes have some entries. I ran all my keys through
Murmur3 hash and they seemed to be evenly distributed.
2. On the nodes that have data, I only see entires in off-heap (Which I
assume is disk). I have 0 entries in heap and 0 in heap memory. Do entries
only get into off-heap memory when they are read?
This is the output from visor for a node:
Total: 419490
Heap: 0
Off-Heap: 419490
Off-Heap Memory: 0
My cluster consists of 8 servers running one node each. Each server has 32GB
RAM and 4 Cores.
All the nodes are started like this
=================
bin/ignite.sh -J-Xmx3g config/dev-cluster-config.xml
=================
My cache consists of
- key: strings of length 9 chars
- Value: POJOs that have some metadata and one large array of integers
(8800 values)
I have 4 data streamers and as they start pushing data, I see some server
nodes throwing OutOfMemoryErrors and the data streamers hangup.
The streamer code is here.
Cache configuration:
==========================================
CacheConfiguration<String, Tile> cacheCfg = new
CacheConfiguration<>(DATA_CACHE_NAME);
cacheCfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheCfg.setCacheMode(CacheMode.PARTITIONED);
cacheCfg.setBackups(2);
cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
cacheCfg.setIndexedTypes(String.class, Tile.class);
==========================================
Cluster-config.xml :
<bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="igniteInstanceName"
value="dev-ignite-cluster-us-east1"/>
<property name="peerClassLoadingEnabled" value="true"/>
<property name="dataStorageConfiguration">
<bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="persistenceEnabled" value="true"/>
<property name="maxSize" value="#{16L * 1024 * 1024
* 1024}"/>
</bean>
</property>
</bean>
</property>
<property name="discoverySpi">
<bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<!-- Aws config goes here-- >
</bean>
</property>
</bean>
=========================================
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Uneven partitioning and data not on off-heap memory.
Posted by smovva <su...@sturfee.com>.
Thanks Denis. That makes sense.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Uneven partitioning and data not on off-heap memory.
Posted by Denis Mekhanikov <dm...@gmail.com>.
Mikael,
You can't configure a node not to use off-heap memory.
It's always used. But you can enable on-heap caching along with it:
https://apacheignite.readme.io/docs/memory-configuration#section-on-heap-caching
So, off-heap memory cannot be disabled.
@smovva, "Off-Heap" shows how many entries are stored in off-heap memory,
and "Off-Heap Memory" – how much off-heap memory is used for it.
Currently calculating how much off-heap memory is used per-cache is not
implemented, so it always shows 0.
And I think, that it won't be implemented since cache memory metrics are
replaces with data region metrics:
https://apacheignite.readme.io/docs/memory-metrics
So, Visor will probably be changed not to show these metrics.
Denis
пн, 2 июл. 2018 г. в 2:37, smovva <su...@sturfee.com>:
> > 2. By default all entries are saved off heap (not disk, heap outside the
> Java heap to avoid GC problems), you can configure it to use heap or off
> heap memory as you want.
>
> This is the output from visor. I'm not completely sure what the difference
> between off-heap and off-heap memory is here. So, I was assuming that
> "off-heap" is disk and "off-heap memory" is what's in off-heap RAM.
> ++++++++++++
> Total: 419490
> Heap: 0
> Off-Heap: 419490
> Off-Heap Memory: 0
> ++++++++++++
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Uneven partitioning and data not on off-heap memory.
Posted by smovva <su...@sturfee.com>.
> 2. By default all entries are saved off heap (not disk, heap outside the
Java heap to avoid GC problems), you can configure it to use heap or off
heap memory as you want.
This is the output from visor. I'm not completely sure what the difference
between off-heap and off-heap memory is here. So, I was assuming that
"off-heap" is disk and "off-heap memory" is what's in off-heap RAM.
++++++++++++
Total: 419490
Heap: 0
Off-Heap: 419490
Off-Heap Memory: 0
++++++++++++
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Uneven partitioning and data not on off-heap memory.
Posted by Mikael <mi...@telia.com>.
1. No idea, have not seen any problems myself.
2. By default all entries are saved off heap (not disk, heap outside the
Java heap to avoid GC problems), you can configure it to use heap or off
heap memory as you want.
Mikael
Den 2018-06-30 kl. 22:43, skrev smovva:
> I have started my cluster and pushed some data into a cache. Cluster details
> are provided below. The cache has persistence enabled.
> I see a couple of issues with how data is cached.
> 1. Out of 8 nodes only 2 nodes have some entries. I ran all my keys through
> Murmur3 hash and they seemed to be evenly distributed.
> 2. On the nodes that have data, I only see entires in off-heap (Which I
> assume is disk). I have 0 entries in heap and 0 in heap memory. Do entries
> only get into off-heap memory when they are read?
> This is the output from visor for a node:
>
> Total: 419490
> Heap: 0
> Off-Heap: 419490
> Off-Heap Memory: 0
>
>
> My cluster consists of 8 servers running one node each. Each server has 32GB
> RAM and 4 Cores.
> All the nodes are started like this
> =================
> bin/ignite.sh -J-Xmx3g config/dev-cluster-config.xml
> =================
>
> My cache consists of
> - key: strings of length 9 chars
> - Value: POJOs that have some metadata and one large array of integers
> (8800 values)
>
> I have 4 data streamers and as they start pushing data, I see some server
> nodes throwing OutOfMemoryErrors and the data streamers hangup.
>
> The streamer code is here.
>
> Cache configuration:
> ==========================================
> CacheConfiguration<String, Tile> cacheCfg = new
> CacheConfiguration<>(DATA_CACHE_NAME);
> cacheCfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
> cacheCfg.setBackups(2);
>
>
> cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
> cacheCfg.setIndexedTypes(String.class, Tile.class);
>
> ==========================================
>
> Cluster-config.xml :
>
> <bean id="ignite.cfg"
> class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="igniteInstanceName"
> value="dev-ignite-cluster-us-east1"/>
> <property name="peerClassLoadingEnabled" value="true"/>
>
> <property name="dataStorageConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataStorageConfiguration">
> <property name="defaultDataRegionConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataRegionConfiguration">
> <property name="persistenceEnabled" value="true"/>
> <property name="maxSize" value="#{16L * 1024 * 1024
> * 1024}"/>
> </bean>
> </property>
> </bean>
> </property>
>
> <property name="discoverySpi">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> <!-- Aws config goes here-- >
> </bean>
> </property>
> </bean>
>
> =========================================
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>