[jira] [Created] (IGNITE-8481) VisorValidateIndexesJob works very slowly in case of many partitions/keys for each partition.

["Alexei Scherbakov (JIRA)" <ji...@apache.org>]

Alexei Scherbakov created IGNITE-8481:
-----------------------------------------

             Summary: VisorValidateIndexesJob works very slowly in case of many partitions/keys for each partition.
                 Key: IGNITE-8481
                 URL: https://issues.apache.org/jira/browse/IGNITE-8481
             Project: Ignite
          Issue Type: Bug
    Affects Versions: 2.5
            Reporter: Alexei Scherbakov
             Fix For: 2.6
         Attachments: ignite.zip, thrdump-server.log

I tried to validate indexes using newly introduced VisorValidateIndexesTask from control.sh and found what on large data set it works very slowly. Process was not finished for 12 hours from start.

Looking through a thread dump I've noticed following problems:

1. ValidateIndexesClosure works not in optimal way by doing btree lookup for each index for each entry of each partition. It should be faster to validate by scanning index tree.

2. Thread dump shows contention on acquiring segment read lock by worker pool-XXX threads, but no obvious reason for holding write lock (no load on grid)

3. org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#partGeneration generates garbage on each page access.

Check attachment for log and thread dump.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)