You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by "Alexand Polyakov (JIRA)" <ji...@apache.org> on 2018/12/05 16:26:00 UTC
[jira] [Created] (IGNITE-10557) Control.sh validate index work long
and broke down
Alexand Polyakov created IGNITE-10557:
-----------------------------------------
Summary: Control.sh validate index work long and broke down
Key: IGNITE-10557
URL: https://issues.apache.org/jira/browse/IGNITE-10557
Project: Ignite
Issue Type: Bug
Components: visor
Affects Versions: 2.6
Reporter: Alexand Polyakov
cluster in the amount of 27Gb
performing validate_indexes took more than 1 hour
and execution failed
{code}
control.sh --cache validate_indexes
Control utility [ver. 2.6]
2018 Copyright(C) Apache Software Foundation
User: pprbusr
--------------------------------------------------------------------------------
Connection to cluster failed.
Error: Failed to perform request (connection failed): /10.117.102.207:11211
You have mail in /var/spool/mail/busr
{code}
analysis of the thread for 40 minutes showed that out of 32 nodes only on 3 nodes were flows ValidateIndexesClosure
at the same time, some threads are blocked
{code}
"pool-55-thread-53" #9255 prio=5 os_prio=0 tid=0x00007eb5a0073800 nid=0xb408 waiting for monitor entry [0x00007eb6554f3000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.ignite.internal.pagemem.PageUtils.getBytes(PageUtils.java:63)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFullRow(CacheDataRowAdapter.java:296)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:159)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:102)
at org.apache.ignite.internal.processors.query.h2.database.H2RowFactory.getRow(H2RowFactory.java:61)
at org.apache.ignite.internal.processors.query.h2.database.H2Tree.createRowFromLink(H2Tree.java:152)
at org.apache.ignite.internal.processors.query.h2.database.io.H2InnerIO.getLookupRow(H2InnerIO.java:60)
at org.apache.ignite.internal.processors.query.h2.database.io.H2InnerIO.getLookupRow(H2InnerIO.java:33)
at org.apache.ignite.internal.processors.query.h2.database.H2Tree.getRow(H2Tree.java:170)
at org.apache.ignite.internal.processors.query.h2.database.H2Tree.getRow(H2Tree.java:47)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.getRow(BPlusTree.java:4524)
at org.apache.ignite.internal.processors.query.h2.database.H2Tree.compare(H2Tree.java:212)
at org.apache.ignite.internal.processors.query.h2.database.H2Tree.compare(H2Tree.java:47)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.compare(BPlusTree.java:4511)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findInsertionPoint(BPlusTree.java:4431)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$1300(BPlusTree.java:90)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:291)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:4858)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:4843)
at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:161)
at org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:332)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findDown(BPlusTree.java:1157)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doFind(BPlusTree.java:1124)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findOne(BPlusTree.java:1091)
at org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.find(H2TreeIndex.java:201)
at org.apache.ignite.internal.visor.verify.ValidateIndexesClosure.processPartition(ValidateIndexesClosure.java:524)
at org.apache.ignite.internal.visor.verify.ValidateIndexesClosure.access$100(ValidateIndexesClosure.java:86)
at org.apache.ignite.internal.visor.verify.ValidateIndexesClosure$2.call(ValidateIndexesClosure.java:394)
at org.apache.ignite.internal.visor.verify.ValidateIndexesClosure$2.call(ValidateIndexesClosure.java:392)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)