You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Sumit Deshinge <su...@gmail.com> on 2022/02/22 02:57:12 UTC
Ever increasing Disk space for a cache in ignite storage directory
Hi,
We are using ignite 2.12 with persistence enabled along with full sync -
replicated mode.
There is a cache whose data ranges from few Kilobytes to few Megabytes.
Once a particular cache entry is worked upon, we remove that entry from
cache. The cache goes through a large number of reads/writes parallely.
But what we have observed is, the disk space utilized by the cache keeps on
increasing. I had expected the disk space to be reused - (deleted cache
entries disk partitions) for new cache entries being inserted , but this is
not happening.
Any reason for this behavior and any solution to avoid such problems?
--
Regards,
Sumit Deshinge
Re: Ever increasing Disk space for a cache in ignite storage directory
Posted by Surinder Mehra <re...@gmail.com>.
Hi,
As per ignite docs, " It is reused when new entries need to be added to the
storage on subsequent write operations. Therefore, the allocated size does
not decrease when you remove entries from the caches ".
Ignite devs can comment more on this.
https://ignite.apache.org/docs/latest/monitoring-metrics/metrics
On Tue, Feb 22, 2022 at 12:48 PM Sumit Deshinge <su...@gmail.com>
wrote:
> Hi Surinder,
>
> We have native persistence enabled, so I guess eviction policies should
> not come into the picture.
> Cache configuration is as follows:
> Atomicity: Transactional
> CacheMode: Replicated
> Rebalance mode: sync
>
> The cache entries are removed using cache APIs : remove and/or removeAll.
>
> This removes cache entries as in I can see cache get api returns nothing
> for the entries removed, but the disk space does not get reduced.
> Even after let's say all entries are removed from cache, disk space used
> for the cache remains as is.
>
> My observation is that the data is deleted from disk after
> cache.destroy(), but I can not do this operation as the data is
> continuously being read/written on the cache.
>
>
> On Tue, Feb 22, 2022 at 9:14 AM Surinder Mehra <re...@gmail.com> wrote:
>
>> Do you manually remove that entry from cache or using eviction/expiration
>> policy.
>>
>> Can you share cache configuration please.
>>
>> This thread might help you(comments)
>>
>>
>> https://stackoverflow.com/questions/48951091/ignite-how-eviction-expiry-and-rebalancing-work-with-external-cachestore
>>
>> On Tue, Feb 22, 2022, 08:27 Sumit Deshinge <su...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> We are using ignite 2.12 with persistence enabled along with full sync -
>>> replicated mode.
>>> There is a cache whose data ranges from few Kilobytes to few Megabytes.
>>> Once a particular cache entry is worked upon, we remove that entry from
>>> cache. The cache goes through a large number of reads/writes parallely.
>>>
>>> But what we have observed is, the disk space utilized by the cache keeps
>>> on increasing. I had expected the disk space to be reused - (deleted cache
>>> entries disk partitions) for new cache entries being inserted , but this is
>>> not happening.
>>>
>>> Any reason for this behavior and any solution to avoid such problems?
>>>
>>> --
>>> Regards,
>>> Sumit Deshinge
>>>
>>>
>
> --
> Regards,
> Sumit Deshinge
>
>
Re: Ever increasing Disk space for a cache in ignite storage directory
Posted by Sumit Deshinge <su...@gmail.com>.
Hi Surinder,
We have native persistence enabled, so I guess eviction policies should not
come into the picture.
Cache configuration is as follows:
Atomicity: Transactional
CacheMode: Replicated
Rebalance mode: sync
The cache entries are removed using cache APIs : remove and/or removeAll.
This removes cache entries as in I can see cache get api returns nothing
for the entries removed, but the disk space does not get reduced.
Even after let's say all entries are removed from cache, disk space used
for the cache remains as is.
My observation is that the data is deleted from disk after cache.destroy(),
but I can not do this operation as the data is continuously being
read/written on the cache.
On Tue, Feb 22, 2022 at 9:14 AM Surinder Mehra <re...@gmail.com> wrote:
> Do you manually remove that entry from cache or using eviction/expiration
> policy.
>
> Can you share cache configuration please.
>
> This thread might help you(comments)
>
>
> https://stackoverflow.com/questions/48951091/ignite-how-eviction-expiry-and-rebalancing-work-with-external-cachestore
>
> On Tue, Feb 22, 2022, 08:27 Sumit Deshinge <su...@gmail.com>
> wrote:
>
>> Hi,
>>
>> We are using ignite 2.12 with persistence enabled along with full sync -
>> replicated mode.
>> There is a cache whose data ranges from few Kilobytes to few Megabytes.
>> Once a particular cache entry is worked upon, we remove that entry from
>> cache. The cache goes through a large number of reads/writes parallely.
>>
>> But what we have observed is, the disk space utilized by the cache keeps
>> on increasing. I had expected the disk space to be reused - (deleted cache
>> entries disk partitions) for new cache entries being inserted , but this is
>> not happening.
>>
>> Any reason for this behavior and any solution to avoid such problems?
>>
>> --
>> Regards,
>> Sumit Deshinge
>>
>>
--
Regards,
Sumit Deshinge
Re: Ever increasing Disk space for a cache in ignite storage directory
Posted by Surinder Mehra <re...@gmail.com>.
Do you manually remove that entry from cache or using eviction/expiration
policy.
Can you share cache configuration please.
This thread might help you(comments)
https://stackoverflow.com/questions/48951091/ignite-how-eviction-expiry-and-rebalancing-work-with-external-cachestore
On Tue, Feb 22, 2022, 08:27 Sumit Deshinge <su...@gmail.com> wrote:
> Hi,
>
> We are using ignite 2.12 with persistence enabled along with full sync -
> replicated mode.
> There is a cache whose data ranges from few Kilobytes to few Megabytes.
> Once a particular cache entry is worked upon, we remove that entry from
> cache. The cache goes through a large number of reads/writes parallely.
>
> But what we have observed is, the disk space utilized by the cache keeps
> on increasing. I had expected the disk space to be reused - (deleted cache
> entries disk partitions) for new cache entries being inserted , but this is
> not happening.
>
> Any reason for this behavior and any solution to avoid such problems?
>
> --
> Regards,
> Sumit Deshinge
>
>