You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by VT <za...@gmail.com> on 2018/02/23 09:02:08 UTC
SSD writes get slower and slower
Hello,
I am trying to test use SSD. But as more data are written, the write
performance drops significantly. Is there something I can do to solve this
problem? Or are there any known issues to this problem that might be fixed
in the upcoming versions? The test was done on a single node, locally. The
total data size is small, not enough to fill half of SSD.
Thanks!
For example,
Loaded 10000 keys 239ms.
Loaded 20000 keys 122ms.
Loaded 30000 keys 141ms.
Loaded 40000 keys 110ms.
...
Loaded 8420000 keys 8068ms.
Loaded 8430000 keys 201ms.
Loaded 8440000 keys 187ms.
Loaded 8450000 keys 162ms.
Loaded 8460000 keys 182ms.
Loaded 8470000 keys 164ms.
Loaded 8480000 keys 171ms.
Loaded 8490000 keys 8475ms.
...
Loaded 84370000 keys 5684ms.
Loaded 84380000 keys 4329ms.
Loaded 84390000 keys 5191ms.
Loaded 84400000 keys 7352ms.
Loaded 84410000 keys 11171ms.
...
Loaded 99940000 keys 13352ms.
Loaded 99950000 keys 28771ms.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: SSD writes get slower and slower
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I also think that switching to LOG_ONLY mode should be good enough.
On Mon, Feb 26, 2018 at 6:58 AM, dmitriy.govorukhin <
dmitriy.govorukhin@gmail.com> wrote:
> Hi,
>
> I guess the problem in "setSwapPath(...)", it is not the path for
> persistence.
>
> Try to do something like this:
>
> storePath="/data/ignite2/swap/"
>
> cfg.setDataStorageConfiguration(
> new DataStorageConfiguration()
> .setWriteThrottlingEnabled(true)
> .setPageSize(4 * 1024)
> .setStoragePath(storePath)
> .setWalPath(storePath + "/wal")
> .setWalArchivePath(storePath + "/archive")
> .setDefaultDataRegionConfiguration(
> new DataRegionConfiguration()
> .setPersistenceEnabled(true)
> .setInitialSize(2L * 1024 * 1024 * 1024)
> .setMaxSize(10L * 1024 * 1024 * 1024)
> )
> );
>
> Are you sure what you need so strong guaranteed? (WALMode.DEFAULT)
> The WALMode.DEFAULT mode provides the strictness of the protection, but it
> is the slowest onе.
> Use LOG_ONLY mode for data loading. More info about wal mode see:
> https://apacheignite.readme.io/docs/write-ahead-log
>
>
>
>
> On 25.02.2018 12:14, VT wrote:
>
>> Hi Stan,
>>
>> The setting is very simple and straightforward, as follows.
>>
>> DataStorageConfiguration dsCfg = new DataStorageConfiguration();
>> dsCfg.setWalMode(WALMode.DEFAULT);
>> dsCfg.setPageSize(4 * 1024);
>> dsCfg.setWriteThrottlingEnabled(true);
>>
>> ..
>>
>> DataRegionConfiguration regionCfg1 = new DataRegionConfiguration();
>> regionCfg1.setName("region_1");
>> regionCfg1.setInitialSize(2L * 1024 * 1024 * 1024);
>> regionCfg1.setMaxSize(10L * 1024 * 1024 * 1024);
>> regionCfg1.setCheckpointPageBufferSize(2L * 1024 * 1024 * 1024);
>> regionCfg1.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
>> regionCfg1.setPersistenceEnabled(true);
>> regionCfg1.setSwapPath("/data/ignite2/swap/"); //SSD
>>
>> ...
>>
>> cacheCfg.setDataRegionName("region_1");
>> cacheCfg.setName(CacheName);
>> cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizati
>> onMode.FULL_ASYNC);
>> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
>> cacheCfg.setCopyOnRead(false);
>>
>> ...
>>
>> I used DataStreammer very simply, like the following.
>>
>> IgniteDataStreamer stmr.addData(key, value);
>>
>> I have tried multiple settings such as perNodeBufferSize,
>> perNodeParallelOperations. Still very slow. Please help. Thanks!
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
Re: SSD writes get slower and slower
Posted by "dmitriy.govorukhin" <dm...@gmail.com>.
Hi,
I guess the problem in "setSwapPath(...)", it is not the path for
persistence.
Try to do something like this:
storePath="/data/ignite2/swap/"
cfg.setDataStorageConfiguration(
new DataStorageConfiguration()
.setWriteThrottlingEnabled(true)
.setPageSize(4 * 1024)
.setStoragePath(storePath)
.setWalPath(storePath + "/wal")
.setWalArchivePath(storePath + "/archive")
.setDefaultDataRegionConfiguration(
new DataRegionConfiguration()
.setPersistenceEnabled(true)
.setInitialSize(2L * 1024 * 1024 * 1024)
.setMaxSize(10L * 1024 * 1024 * 1024)
)
);
Are you sure what you need so strong guaranteed? (WALMode.DEFAULT)
The WALMode.DEFAULT mode provides the strictness of the protection, but
it is the slowest onе.
Use LOG_ONLY mode for data loading. More info about wal mode see:
https://apacheignite.readme.io/docs/write-ahead-log
On 25.02.2018 12:14, VT wrote:
> Hi Stan,
>
> The setting is very simple and straightforward, as follows.
>
> DataStorageConfiguration dsCfg = new DataStorageConfiguration();
> dsCfg.setWalMode(WALMode.DEFAULT);
> dsCfg.setPageSize(4 * 1024);
> dsCfg.setWriteThrottlingEnabled(true);
>
> ..
>
> DataRegionConfiguration regionCfg1 = new DataRegionConfiguration();
> regionCfg1.setName("region_1");
> regionCfg1.setInitialSize(2L * 1024 * 1024 * 1024);
> regionCfg1.setMaxSize(10L * 1024 * 1024 * 1024);
> regionCfg1.setCheckpointPageBufferSize(2L * 1024 * 1024 * 1024);
> regionCfg1.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
> regionCfg1.setPersistenceEnabled(true);
> regionCfg1.setSwapPath("/data/ignite2/swap/"); //SSD
>
> ...
>
> cacheCfg.setDataRegionName("region_1");
> cacheCfg.setName(CacheName);
> cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_ASYNC);
> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
> cacheCfg.setCopyOnRead(false);
>
> ...
>
> I used DataStreammer very simply, like the following.
>
> IgniteDataStreamer stmr.addData(key, value);
>
> I have tried multiple settings such as perNodeBufferSize,
> perNodeParallelOperations. Still very slow. Please help. Thanks!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: SSD writes get slower and slower
Posted by VT <za...@gmail.com>.
Hi Stan,
The setting is very simple and straightforward, as follows.
DataStorageConfiguration dsCfg = new DataStorageConfiguration();
dsCfg.setWalMode(WALMode.DEFAULT);
dsCfg.setPageSize(4 * 1024);
dsCfg.setWriteThrottlingEnabled(true);
..
DataRegionConfiguration regionCfg1 = new DataRegionConfiguration();
regionCfg1.setName("region_1");
regionCfg1.setInitialSize(2L * 1024 * 1024 * 1024);
regionCfg1.setMaxSize(10L * 1024 * 1024 * 1024);
regionCfg1.setCheckpointPageBufferSize(2L * 1024 * 1024 * 1024);
regionCfg1.setPageEvictionMode(DataPageEvictionMode.RANDOM_2_LRU);
regionCfg1.setPersistenceEnabled(true);
regionCfg1.setSwapPath("/data/ignite2/swap/"); //SSD
...
cacheCfg.setDataRegionName("region_1");
cacheCfg.setName(CacheName);
cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_ASYNC);
cacheCfg.setCacheMode(CacheMode.PARTITIONED);
cacheCfg.setCopyOnRead(false);
...
I used DataStreammer very simply, like the following.
IgniteDataStreamer stmr.addData(key, value);
I have tried multiple settings such as perNodeBufferSize,
perNodeParallelOperations. Still very slow. Please help. Thanks!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: SSD writes get slower and slower
Posted by Stanislav Lukyanov <st...@gmail.com>.
Hi,
Just to confirm – you’re talking about Ignite’s native persistence, right?
Can you share the code and configs you use in your testing?
Thanks,
Stan
From: VT
Sent: 23 февраля 2018 г. 12:02
To: user@ignite.apache.org
Subject: SSD writes get slower and slower
Hello,
I am trying to test use SSD. But as more data are written, the write
performance drops significantly. Is there something I can do to solve this
problem? Or are there any known issues to this problem that might be fixed
in the upcoming versions? The test was done on a single node, locally. The
total data size is small, not enough to fill half of SSD.
Thanks!
For example,
Loaded 10000 keys 239ms.
Loaded 20000 keys 122ms.
Loaded 30000 keys 141ms.
Loaded 40000 keys 110ms.
...
Loaded 8420000 keys 8068ms.
Loaded 8430000 keys 201ms.
Loaded 8440000 keys 187ms.
Loaded 8450000 keys 162ms.
Loaded 8460000 keys 182ms.
Loaded 8470000 keys 164ms.
Loaded 8480000 keys 171ms.
Loaded 8490000 keys 8475ms.
...
Loaded 84370000 keys 5684ms.
Loaded 84380000 keys 4329ms.
Loaded 84390000 keys 5191ms.
Loaded 84400000 keys 7352ms.
Loaded 84410000 keys 11171ms.
...
Loaded 99940000 keys 13352ms.
Loaded 99950000 keys 28771ms.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/