[jira] [Commented] (IGNITE-8037) DynamicColumnsConcurrentTransactionalPartitionedSelfTest#testClientReconnectWithNonDynamicCacheRestart is flaky

["Sergey Chugunov (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-8037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16415599#comment-16415599 ] 

Sergey Chugunov commented on IGNITE-8037:
-----------------------------------------

I found out the cause of test failure but haven't fixed it yet.

Here is what happens in the test when it fails: 
# When client node discovers that it disconnected it tries to stop all caches started on it. 
# For some reason H2 Database cannot find schema for a cache and throws an exception.
# Exception is only logged, no further actions are executed to recover from the situation.
# When connectivity to the grid restores client node tries to start all configured caches, but situation with lacking schema repeats.
# Exchange triggered by client fails so no RECONNECT event is triggered; therefore test fails with outcome from description.

The root cause of the issue lies in step#2 - one needs to figure out what happens with schema registry inside H2 and fix it.

> DynamicColumnsConcurrentTransactionalPartitionedSelfTest#testClientReconnectWithNonDynamicCacheRestart is flaky
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-8037
>                 URL: https://issues.apache.org/jira/browse/IGNITE-8037
>             Project: Ignite
>          Issue Type: Sub-task
>            Reporter: Sergey Chugunov
>            Assignee: Sergey Chugunov
>            Priority: Major
>
> Test fails on TC as well as locally (although local runs reproduce failure rarely: 1-3 failed tests for 100 runs).
> There are suspicious errors in logs:
> {noformat}
> [2018-03-23 18:55:19,371][ERROR][exchange-worker-#630%index.DynamicColumnsConcurrentTransactionalPartitionedSelfTest4%][GridQueryProcessor] Failed to clear indexing on cache unregister (will ignore): idx
> class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to set schema for DB connection for thread [schema=idx]
> at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.connectionForThread(IgniteH2Indexing.java:542)
> at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.dropTable(IgniteH2Indexing.java:705)
> at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.unregisterCache(IgniteH2Indexing.java:2794)
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStop0(GridQueryProcessor.java:1684)
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.registerCache0(GridQueryProcessor.java:1630)
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:799)
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:860)
> at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1161)
> at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1902)
> at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:1768)
> at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:784)
> at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:666)
> at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2344)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.h2.jdbc.JdbcSQLException: Schema "idx" not found; SQL statement:
> SET SCHEMA "idx" [90079-195]
> at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
> at org.h2.message.DbException.get(DbException.java:179)
> at org.h2.message.DbException.get(DbException.java:155)
> at org.h2.engine.Database.getSchema(Database.java:1755)
> at org.h2.command.dml.Set.update(Set.java:408)
> at org.h2.command.CommandContainer.update(CommandContainer.java:101)
> at org.h2.command.Command.executeUpdate(Command.java:260)
> at org.h2.jdbc.JdbcStatement.executeUpdateInternal(JdbcStatement.java:137)
> at org.h2.jdbc.JdbcStatement.executeUpdate(JdbcStatement.java:122)
> at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.connectionForThread(IgniteH2Indexing.java:534)
> ... 14 more{noformat}
> Test fails with the following error:
> {noformat}
> junit.framework.AssertionFailedError: Failed to wait for disconnect/reconnect event.
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.TestCase.fail(TestCase.java:227)
> at org.apache.ignite.internal.IgniteClientReconnectAbstractTest.waitReconnectEvent(IgniteClientReconnectAbstractTest.java:120)
> at org.apache.ignite.internal.IgniteClientReconnectAbstractTest.reconnectClientNodes(IgniteClientReconnectAbstractTest.java:297)
> at org.apache.ignite.internal.IgniteClientReconnectAbstractTest.reconnectClientNode(IgniteClientReconnectAbstractTest.java:238)
> at org.apache.ignite.internal.processors.cache.index.DynamicColumnsAbstractConcurrentSelfTest.reconnectClientNode(DynamicColumnsAbstractConcurrentSelfTest.java:804)
> at org.apache.ignite.internal.processors.cache.index.DynamicColumnsAbstractConcurrentSelfTest.checkClientReconnect(DynamicColumnsAbstractConcurrentSelfTest.java:781)
> at org.apache.ignite.internal.processors.cache.index.DynamicColumnsAbstractConcurrentSelfTest.testClientReconnectWithNonDynamicCacheRestart(DynamicColumnsAbstractConcurrentSelfTest.java:750)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2002)
> at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:133)
> at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1917)
> at java.lang.Thread.run(Thread.java:745){noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)