You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Prasad Pillala <Pr...@blueyonder.com> on 2020/10/05 14:53:41 UTC
Eviction policy enablement leads to ignite cluster blocking, it does
not use pages from freelist
Hi,
evictDataPage() always leads to ignite cluster blocked, due to some reason.
This method does not seem to consider the freelist, which is still have some/many pages available. But evictDataPage() still trying to evict few entries from filled pages, and after sometime (in few mins, after it reached evictionThreshold memory); it is not getting any pages/entries to evict. It started reporting "Too many failed attempts to evict page: 30".
My igniteconfiguration as follows:
DataRegionConfiguration
dataRegionConfig.setMaxSize(8L * 1024 * 1024 * 1024)//8GB
dataRegionConfig.setPageEvictionMode(DataPageEvictionMode.RANDOM_LRU)//tried
LRU2 as well
...
igniteDataCfg.setPageSize(pageSizeKB)//16KB
Ignite version - 2.8.0
Using only Off-Heap for caching. DataRegion persistence is disabled, as we have 3rd party persistence configured with read-through & write-through enabled.
When I tried different evictionThreshold, still got the same result. Not sure, what is the problem with my configuration.
Many thanks in advance for your help.
Stay ahead of today's supply chain complexities with Luminate Control Tower. Start a free 30-day trial here!<https://now.blueyonder.com/control-tower-trial.html>
Re: Eviction policy enablement leads to ignite cluster blocking, it
does not use pages from freelist
Posted by pvprsd <pr...@blueyonder.com>.
Hi Evgenii,
Later investigations, we found that this is happening due to the near-cache
"rdrs" not released from entries i.e. GridDhtCacheEntry objects.
When near-cache entry is getting evicted from a ignite-client node; it is
not clearing the rdrs from Server. Due to these rdrs not empty, the entries
are not getting evicted; when eviction started.
As per NearCache documentation, the rdrs will be maintained to track the
client-nodes, so that entry changes will be posted to these near-cache
entries. But what I have observed is, the rdrs are not getting cleared; when
the local node evicts the entry. I have confirmed the local-node eviction by
monitoring the local-node metrics.
When we turned-off the near-cache completely, the eviction working well, as
the rdrs are not blocking the eviction this time.
Is there any specific configuration required to be enabled to clear the rdrs
from GridDhtCacheEntry objects?
Thanks,
Prasad
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Eviction policy enablement leads to ignite cluster blocking, it
does not use pages from freelist
Posted by ezhuravlev <e....@gmail.com>.
Hi,
From logs, I see that all server nodes are pinging node on host 10.180.48.14
all the time. It can be caused by the lack of the connection between part of
the nodes(for example, if node 10.180.48.14 can't directly communicate with
client nodes). I think that client nodes might ask server nodes to check if
node on 10.180.48.14 is still alive. I would recommend checking connectivity
between all the machines in the cluster, including client machines.
Best Regards,
Evgenii
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Eviction policy enablement leads to ignite cluster blocking, it
does not use pages from freelist
Posted by pvprsd <pr...@blueyonder.com>.
Hi Evgenii,
We are using EntryProcessor to update/insert objects into cache, in addition
to GET cache object calls. Multiple services are making these calls using
IgniteClients. We have enabled NearCache on Ignite Clients.
Size of the objects: We will figure out today, and we will post it here
today.
Share Logs: I am attaching the logs here. Currently I am running 4 ignite
server nodes, out of 1 ignite node reported this problem.
Reproducer: I will try to create it.
Many thanks for your help.
ignite3.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t3000/ignite3.zip>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Eviction policy enablement leads to ignite cluster blocking, it
does not use pages from freelist
Posted by Evgenii Zhuravlev <e....@gmail.com>.
Hi Prasad,
What operations do you run on the cluster? What is the size of objects? Is
it possible to share full logs from nodes? Do you have some kind of small
reproducer for this issue? It would be really helpful.
Thanks,
Evgenii
пн, 5 окт. 2020 г. в 07:53, Prasad Pillala <Pr...@blueyonder.com>:
> Hi,
>
>
>
> evictDataPage() always leads to ignite cluster blocked, due to some reason.
>
> This method does not seem to consider the freelist, which is still have
> some/many pages available. But evictDataPage() still trying to evict few
> entries from filled pages, and after sometime (in few mins, after it
> reached evictionThreshold memory); it is not getting any pages/entries to
> evict. It started reporting "Too many failed attempts to evict page: 30".
>
>
>
> My igniteconfiguration as follows:
>
> DataRegionConfiguration
>
> dataRegionConfig.setMaxSize(8L * 1024 * 1024 * 1024)//8GB
>
>
>
>
> dataRegionConfig.setPageEvictionMode(DataPageEvictionMode.RANDOM_LRU)//tried
>
> LRU2 as well
>
> ...
>
> igniteDataCfg.setPageSize(pageSizeKB)//16KB
>
>
>
> Ignite version - 2.8.0
>
>
>
> Using only Off-Heap for caching. DataRegion persistence is disabled, as we
> have 3rd party persistence configured with read-through & write-through
> enabled.
>
>
>
> When I tried different evictionThreshold, still got the same result. Not
> sure, what is the problem with my configuration.
>
>
>
> Many thanks in advance for your help.
>
>
>
>
>
> *Stay ahead of today’s supply chain complexities with Luminate Control
> Tower. Start a free 30-day trial here!
> <https://now.blueyonder.com/control-tower-trial.html> *
>