[jira] [Commented] (IGNITE-10684) Memory leak in persistent IgniteSet

["Pavel Pereslegin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721325#comment-16721325 ] 

Pavel Pereslegin commented on IGNITE-10684:
-------------------------------------------

[~zafiglon], could you please re-check results for IgniteQueue, Just replace
{code:java}
final IgniteSet<Object> set = ignite
                    .set(name, setConfig);
{code}
with
{code:java}
final IgniteQueue<Object> set = ignite.queue(name, 0, setConfig);
{code}

I see the same leak, that you describe. Even if you will create atomics (AtomicSequence, AtomicLong and so on) in different cache-groups you should see heap leak (but not so fast).

The main problem here at the line:
{code:java}
setConfig.setGroupName(name);
{code}

Each datastructure hold self state in special system-atomics cache, this cache will be created for each *new* cache group.
Datastructure collections (set and queue) uses additional system cache to store its elements.

Datastrctures from same *cache group* shares *single cache* for multiple instances.

So I see some workarounds for your case:
1. Don't change group name each time in collection configuration (recommended).
2. Properly destroy system caches, for your *ATOMIC LOCAL* cache mode with *0* backups code should look like (*just for testing, may break after update and will break after cache mode change*):
{code:java}
ignite.destroyCache("datastructures_ATOMIC_LOCAL_0@" + name);
ignite.destroyCache("ignite-sys-atomic-cache@" + name);
{code}

----
From the developers' point of view, I think it's need to investigate possibility to not create *ignite-sys-atomic-cache* cache in each cache group (use one if possible) and/or implement something like cleanup for shared datastructure caches.



> Memory leak in persistent IgniteSet
> -----------------------------------
>
>                 Key: IGNITE-10684
>                 URL: https://issues.apache.org/jira/browse/IGNITE-10684
>             Project: Ignite
>          Issue Type: Bug
>          Components: data structures, persistence
>    Affects Versions: 2.7
>            Reporter: Alexey Belov
>            Priority: Critical
>              Labels: Ignite, igniteset, memory-leak, persistant
>             Fix For: None
>
>         Attachments: IgniteManagerTest.start - JProfiler 9.2.1 2.jpg, IgniteManagerTest.start - JProfiler 9.2.1 3.jpg, IgniteManagerTest.start - JProfiler 9.2.1.jpg
>
>
> Hello. I have found a leak in IgniteSet with using persistence. Here is my Unit Test:
> {code:java}
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteSet;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.configuration.*;
> import org.junit.Test;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
> import java.util.UUID;
> import java.util.concurrent.LinkedBlockingQueue;
> import java.util.concurrent.ThreadPoolExecutor;
> import java.util.concurrent.TimeUnit;
> /**
>  * @author Alexey Belov
>  */
> public class IgniteManagerTest {
>     protected final Logger log = LoggerFactory.getLogger(IgniteManagerTest.class);
>     private ThreadPoolExecutor ex = new ThreadPoolExecutor(8, 8,
>             0L, TimeUnit.MILLISECONDS,
>             new LinkedBlockingQueue<Runnable>());
>     @Test
>     public void start() throws Exception {
>         final IgniteConfiguration cfg = new IgniteConfiguration();
>         final DataStorageConfiguration dataStorageConfiguration = new DataStorageConfiguration();
>         final String igniteStorageDir = "g:\\work\\garbage\\igniteTest\\" + UUID.randomUUID().toString();
>         System.out.println(igniteStorageDir);
>         dataStorageConfiguration.setStoragePath(igniteStorageDir);
>         final DataRegionConfiguration defaultDataRegionConfiguration = dataStorageConfiguration
>                 .getDefaultDataRegionConfiguration();
>         defaultDataRegionConfiguration.setEvictionThreshold(0.9);
>         defaultDataRegionConfiguration.setMetricsEnabled(true);
>         defaultDataRegionConfiguration.setPersistenceEnabled(true);
>         dataStorageConfiguration.setWalMode(WALMode.NONE);
>         cfg.setDataStorageConfiguration(dataStorageConfiguration);
>         final Ignite ignite = Ignition.start(cfg);
>         ignite.cluster().active(true);
>         while (true) {
>             if (ex.getQueue().size() < 8) {
>                 System.out.println("added task " + ex.getQueue().size() + " " + ex.getActiveCount());
>                 ex.execute(() -> runQueues(ignite));
>             }
>             Thread.sleep(1000);
>         }
>     }
>     private void qu() {
>     }
>     private void runQueues(Ignite ignite) {
>         for (int j = 0; j < 10; j++) {
>             final CollectionConfiguration setConfig = new CollectionConfiguration();
>             setConfig.setCacheMode(CacheMode.LOCAL);
>             setConfig.setBackups(0);
>             final String name = "set-" + j + UUID.randomUUID().toString();
>             setConfig.setGroupName(name);
>             final IgniteSet<Object> set = ignite
>                     .set(name, setConfig);
>             final int i1 = 1000;
>             for (int i = 0; i < i1; i++) {
>                 final String elem1 = UUID.randomUUID().toString();
>                 set.add(elem1);
>             }
>             log.info(j + "write");
>             set.clear();
>             set.close();
>             ignite.destroyCache(name);
>         }
>         log.info("Finish!");
>     }
> }
> {code}
> See the attached screenshots from JProfiler.
> I think, that it should not be like this, because i clear the set and memory should be freed.
> If i launch this test with queue it works fine, memory becomes free after some time.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)