You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Alexey Belov <a....@a1-systems.com> on 2018/11/21 06:31:15 UTC
How to force ignite clear disk space
Hello.
I use ignite as persistent queues. I create queues dynamically, so every day i can create up to 50 quques and stream about 5 million records in each. When i don't need this queue anymore, I want to clear my disk space, but it is not happenning. Here is some example code.
final IgniteConfiguration cfg = new IgniteConfiguration();
final DataStorageConfiguration dataStorageConfiguration = new DataStorageConfiguration();
final String igniteStorageDir = "c:\\work\\garbage\\igniteTest\\5";
System.out.println(igniteStorageDir);
dataStorageConfiguration.setStoragePath(igniteStorageDir);
final DataRegionConfiguration defaultDataRegionConfiguration = dataStorageConfiguration
.getDefaultDataRegionConfiguration();
// defaultDataRegionConfiguration.setMaxSize(igniteDataRegionMaxSize);
defaultDataRegionConfiguration.setEvictionThreshold(0.9);
defaultDataRegionConfiguration.setMetricsEnabled(true);
defaultDataRegionConfiguration.setPersistenceEnabled(true);
dataStorageConfiguration.setWalMode(WALMode.NONE);
// dataStorageConfiguration.setWalPath(igniteStorageDir + "/wal");
cfg.setDataStorageConfiguration(dataStorageConfiguration);
final Ignite ignite = Ignition.start(cfg);
ignite.cluster().active(true);
for (int j = 0; j < 10; j++) {
final CollectionConfiguration queueConfig = new CollectionConfiguration();
queueConfig.setCacheMode(CacheMode.LOCAL);
// queueConfig.setBackups(1);
final String queueName = "queue-" + j;
queueConfig.setGroupName(queueName);
final IgniteQueue<Object> queue = ignite
.queue(queueName, 0, queueConfig);
final int i1 = 1000;
for (int i = 0; i < i1; i++) {
final String elem1 = UUID.randomUUID().toString();
queue.put(elem1);
}
System.out.println(j + "queue write");
for (int i = 0; i < i1; i++) {
final Object poll = queue.poll();
}
System.out.println(j + "queue read");
// System.out.println("read complete");
queue.clear();
queue.close();
ignite.destroyCache(queueName);
}
System.out.println("write complete");
ignite.cluster().active(false);
Ignition.stopAll(false);
After this test is complete, I see all this queues on disk, they are not cleared. Is there a way to force ignite to clear disk space after queue becomes empty?
Re: How to force ignite clear disk space
Posted by aealexsandrov <ae...@gmail.com>.
Hi,
I thought that it was fixed here:
https://issues.apache.org/jira/browse/IGNITE-8021
But possible you face another issue.
No 2.7 isn't released now but you can use nightly builds to check it or
build from the source code.
In case if you can reproduce it then you may file the new Jira ticket here
https://issues.apache.org/jira/projects/IGNITE/summary.
Don't forget to attach into ticket:
1)Code of reproducer
2)The configuration of the nodes
3)Logs from the nodes
BR,
Andrei
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to force ignite clear disk space
Posted by Alexey Belov <a....@a1-systems.com>.
Andrey, If it is a bug, may be we should create an issue for this ? Or there is already one?
By the way, when will 2.7 be released ?
________________________________
От: aealexsandrov <ae...@gmail.com>
Отправлено: 22 ноября 2018 г. 14:29:25
Кому: user@ignite.apache.org
Тема: Re: How to force ignite clear disk space
Thank you for details.
I will check it in 2.6 release but I think that it should be fixed in 2.7.
BR,
Andrei
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to force ignite clear disk space
Posted by aealexsandrov <ae...@gmail.com>.
Thank you for details.
I will check it in 2.6 release but I think that it should be fixed in 2.7.
BR,
Andrei
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to force ignite clear disk space
Posted by Alexey Belov <a....@a1-systems.com>.
WAL is disabled, as you can see in config:
dataStorageConfiguration.setWalMode(WALMode.NONE);
My ignite version is 2.6
That is what I have in directory after completing the test:
[cid:3ce7ec54-a3be-4eda-b319-c184a1ceabf9]
________________________________
От: aealexsandrov <ae...@gmail.com>
Отправлено: 21 ноября 2018 г. 19:55:29
Кому: user@ignite.apache.org
Тема: Re: How to force ignite clear disk space
Hi,
I think that you face the issue related to the next ticket:
https://gridgain.freshdesk.com/helpdesk/tickets/9049
After destroying of the cache remains some files that contain the
configuration of the cache. But it should be already fixed in the latest
version of Ignite.
However, the files that contain the cache data should be removed. You should
check the size before destroying and after it.
Another issue is WAL. It could grow without limit in case if you don't
configure the WAL history size or turn WAL logging off for some caches.
You can read about it here:
https://apacheignite.readme.io/docs/write-ahead-log
In case if you see partitions files (.bin) after destroying of the cache
then it could be an issue. What is your Ignite version?
BR,
Andrei
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: How to force ignite clear disk space
Posted by aealexsandrov <ae...@gmail.com>.
Hi,
I think that you face the issue related to the next ticket:
https://gridgain.freshdesk.com/helpdesk/tickets/9049
After destroying of the cache remains some files that contain the
configuration of the cache. But it should be already fixed in the latest
version of Ignite.
However, the files that contain the cache data should be removed. You should
check the size before destroying and after it.
Another issue is WAL. It could grow without limit in case if you don't
configure the WAL history size or turn WAL logging off for some caches.
You can read about it here:
https://apacheignite.readme.io/docs/write-ahead-log
In case if you see partitions files (.bin) after destroying of the cache
then it could be an issue. What is your Ignite version?
BR,
Andrei
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/