You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ignite.apache.org by Andrey Kornev <an...@hotmail.com> on 2017/11/06 22:28:16 UTC

Data region's data page size

Hello,

What are the considerations for choosing the best data page size? It looks like there are only four possible values (1K, 4K, 8K and 16K) to choose from, and I don't immediately see why one would want to use anything other than 16K?

Is it because the value lookup within a page is linear and therefore, I want to keep the number of values that fit on a single page sufficiently low?

Please advise!

Thanks
Andrey

Re: Data region's data page size

Posted by Denis Magda <dm...@apache.org>.

Yes, you should if average object size exceeds 4KB because otherwise Ignite will spend more time spreading and maintaining a single object across several pages. 

—
Denis

> On Nov 7, 2017, at 11:14 AM, Andrey Kornev <an...@hotmail.com> wrote:
> 
> Thanks Ivan! A follow-up question, if you don't mind. 
> 
> Should I consider changing the default 4K page size to 8K (or possibly to 16K) if the average size of my cache objects is, say, 12K, with max size up to 100K?
> 
> Thanks
> Andrey
> 
> From: Ivan Rakov <iv...@gmail.com>
> Sent: Tuesday, November 7, 2017 8:22 AM
> To: user@ignite.apache.org
> Subject: Re: Data region's data page size
>  
> Hello Andrey,
> 
> Short answer: just keep default 4k page size, it will work fine unless you have very specific hardware.
> 
> Too small page size causes write amplification on hardware level: https://www.cactus-tech.com/resources/blog/details/write-amplification-as-seen-in-solid-state-drives <https://www.cactus-tech.com/resources/blog/details/write-amplification-as-seen-in-solid-state-drives>
> On the other hand, too big page size causes write amplification on Ignite level. Whole 16k page will be persisted if you have changed only few bytes.
> 
> You may also want to check our durable memory tuning guide for more details:https://apacheignite.readme.io/docs/durable-memory-tuning <https://apacheignite.readme.io/docs/durable-memory-tuning>
> Best Regards,
> Ivan Rakov
> 
> On 07.11.2017 1:28, Andrey Kornev wrote:
>> Hello,
>> 
>> What are the considerations for choosing the best data page size? It looks like there are only four possible values (1K, 4K, 8K and 16K) to choose from, and I don't immediately see why one would want to use anything other than 16K?
>> 
>> Is it because the value lookup within a page is linear and therefore, I want to keep the number of values that fit on a single page sufficiently low?
>> 
>> Please advise!
>> 
>> Thanks
>> Andrey

Re: Data region's data page size

Posted by Andrey Kornev <an...@hotmail.com>.

Thanks Ivan! A follow-up question, if you don't mind.

Should I consider changing the default 4K page size to 8K (or possibly to 16K) if the average size of my cache objects is, say, 12K, with max size up to 100K?

Thanks
Andrey

________________________________
From: Ivan Rakov <iv...@gmail.com>
Sent: Tuesday, November 7, 2017 8:22 AM
To: user@ignite.apache.org
Subject: Re: Data region's data page size


Hello Andrey,


Short answer: just keep default 4k page size, it will work fine unless you have very specific hardware.


Too small page size causes write amplification on hardware level: https://www.cactus-tech.com/resources/blog/details/write-amplification-as-seen-in-solid-state-drives

On the other hand, too big page size causes write amplification on Ignite level. Whole 16k page will be persisted if you have changed only few bytes.


You may also want to check our durable memory tuning guide for more details: https://apacheignite.readme.io/docs/durable-memory-tuning

Best Regards,
Ivan Rakov


On 07.11.2017 1:28, Andrey Kornev wrote:
Hello,

What are the considerations for choosing the best data page size? It looks like there are only four possible values (1K, 4K, 8K and 16K) to choose from, and I don't immediately see why one would want to use anything other than 16K?

Is it because the value lookup within a page is linear and therefore, I want to keep the number of values that fit on a single page sufficiently low?

Please advise!

Thanks
Andrey

Re: Data region's data page size

Posted by Ivan Rakov <iv...@gmail.com>.

Hello Andrey,


Short answer: just keep default 4k page size, it will work fine unless 
you have very specific hardware.


Too small page size causes write amplification on hardware level: 
https://www.cactus-tech.com/resources/blog/details/write-amplification-as-seen-in-solid-state-drives

On the other hand, too big page size causes write amplification on 
Ignite level. Whole 16k page will be persisted if you have changed only 
few bytes.


You may also want to check our durable memory tuning guide for more 
details: https://apacheignite.readme.io/docs/durable-memory-tuning

Best Regards,
Ivan Rakov


On 07.11.2017 1:28, Andrey Kornev wrote:
> Hello,
>
> What are the considerations for choosing the best data page size? It 
> looks like there are only four possible values (1K, 4K, 8K and 16K) to 
> choose from, and I don't immediately see why one would want to use 
> anything other than 16K?
>
> Is it because the value lookup within a page is linear and 
> therefore, I want to keep the number of values that fit on a single 
> page sufficiently low?
>
> Please advise!
>
> Thanks
> Andrey