Setting Cache Size using Max Records

[Victor <vi...@gmail.com>]

Hi,

I am looking to set the cache size for off-heap via number of max records
instead of max bytes. Similar to how LRU eviction policy for on-heap can set
it.

Haven't found anything around that. Is that doable, if yes, how do i go
about it?

Thanks,
Vic



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Setting Cache Size using Max Records

[Victor <vi...@gmail.com>]

From what i have read so far, seems like if i have 2 caches each to be
configured with 50MB, then i need to define 2 identical regions with unique
names and apply to each of the caches.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Setting Cache Size using Max Records

[Victor <vi...@gmail.com>]

Thanks Denis this helps.

Btw, one thing to check, if you create a 25MB region and multiple cache's
are associated to that region, do all the caches share the region capacity
or just share the configuration but each cache will be allocated a 25MB
capacity?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Setting Cache Size using Max Records

[Denis Magda <dm...@apache.org>]

1. There is no any public API for that but you can follow a workaround
suggested here
<https://stackoverflow.com/questions/49780044/apache-ignite-binaryobject-bloat>.
If you have a POJO class of your domain object, then create an instance,
convert it to BinaryObjectImpl and get the array size (as suggested on
StackOverflow).

2. No, you have to set the size in advance. However, the combination of
initial and max size might do the trick for you. Use the initial size for
after you calculate the value for an expected number of records the cache
has to store while the max size can give you an extra space to deal with
the calculation/estimations errors.

-
Denis


On Tue, Jun 30, 2020 at 9:08 PM Victor <vi...@gmail.com> wrote:

> Yes, user upfront defines a config and the idea indeed is to convert the
> number of records to total bytes.
>
> Followup around this. Given i want to calculate the exact row entry size
> upfront, even before the data is added. Couple of queries,
>
> 1. Is there a way(api) just based on the Cache (table), be able to inspect
> the capacity a row can hold based on the column datatypes?
> 2. Can the Dataregion be set post the cache is created?
>
> Thanks,
> Victor
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Setting Cache Size using Max Records

[Victor <vi...@gmail.com>]

Yes, user upfront defines a config and the idea indeed is to convert the
number of records to total bytes. 

Followup around this. Given i want to calculate the exact row entry size
upfront, even before the data is added. Couple of queries,

1. Is there a way(api) just based on the Cache (table), be able to inspect
the capacity a row can hold based on the column datatypes?
2. Can the Dataregion be set post the cache is created?

Thanks,
Victor



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Setting Cache Size using Max Records

[Denis Magda <dm...@apache.org>]

Victor,

Is it correct to assume that the users define this configuration first and
only then you span up an Ignite cluster for them? If it's so, then you can
translate a user-defined number of records to final data region size. The
formula might be as follows - "data_region_size =
provided_number_of_records * avg_record_size *
avg_ignite_overhead_per_record * K" where:

   - avg_record_size - you said you know it,
   - avg_ignite_overhead_per_record - the overhead is 200 bytes on average
   (refer to the callout in this section
   https://apacheignite.readme.io/docs/capacity-planning#calculating-memory-usage
   )
   - K - that's extra overhead. You might need to take into account a
   memory space for indexes, backup records:
   https://apacheignite.readme.io/docs/capacity-planning#memory-capacity-planning-example


This spreadsheet calculator can be helpful with K:
https://apacheignite.readme.io/docs/capacity-planning#example

-
Denis


On Thu, Jun 25, 2020 at 9:44 AM Victor <vi...@gmail.com> wrote:

> Iiya/ Denis,
>
> I am aware of the Data Region with max size as bytes. I was looking for way
> to do this via number of records.
>
> Anyway, seems there is no way in Ignite to do this today.
>
> I am looking at couple of other options to achieve this, for that same i am
> looking at a way to calculate the storage size per record in cache/table.
> Is
> there an api available to retrieve that?
>
> Thanks,
> Vic
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Setting Cache Size using Max Records

[Victor <vi...@gmail.com>]

Iiya/ Denis,

I am aware of the Data Region with max size as bytes. I was looking for way
to do this via number of records.

Anyway, seems there is no way in Ignite to do this today.

I am looking at couple of other options to achieve this, for that same i am
looking at a way to calculate the storage size per record in cache/table. Is
there an api available to retrieve that?

Thanks,
Vic



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Setting Cache Size using Max Records

[Ilya Kasnacheev <il...@gmail.com>]

Hello!

You can have a data region configured for this specific cache, with max
size (in bytes) and page eviction on.

Regards,
-- 
Ilya Kasnacheev


чт, 25 июн. 2020 г. в 07:16, Victor <vi...@gmail.com>:

> Hi Denis,
>
> For our product we are using an in-house solution where we expose the size
> in records. Now we are looking at replacing it with Ignite. So we wanted to
> make the experience seamless and let the end user's continue to set the
> same
> configuration they are familiar with, rather than adding a new learning
> curve.
>
> Does any of the expiry policies except record size?
>
> Thanks,
> Vic
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Setting Cache Size using Max Records

[Victor <vi...@gmail.com>]

Hi Denis,

For our product we are using an in-house solution where we expose the size
in records. Now we are looking at replacing it with Ignite. So we wanted to
make the experience seamless and let the end user's continue to set the same
configuration they are familiar with, rather than adding a new learning
curve.

Does any of the expiry policies except record size?

Thanks,
Vic



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Setting Cache Size using Max Records

[Denis Magda <dm...@apache.org>]

Hi Vic,

That’s unsupported for the off-heap memory. You can only limit memory usage
based on the size in bytes.

Please share more details of why this type of records eviction is needed.
We might suggest an alternate solution.

Denis

On Wednesday, June 24, 2020, Victor <vi...@gmail.com> wrote:

> Hi,
>
> I am looking to set the cache size for off-heap via number of max records
> instead of max bytes. Similar to how LRU eviction policy for on-heap can
> set
> it.
>
> Haven't found anything around that. Is that doable, if yes, how do i go
> about it?
>
> Thanks,
> Vic
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>


-- 
-
Denis