You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Pavel Pereslegin (JIRA)" <ji...@apache.org> on 2018/10/27 20:01:00 UTC
[jira] [Comment Edited] (IGNITE-9858) [Test Failed]
SystemCacheNotConfiguredTest#test flaky fails on TC (timeout).
[ https://issues.apache.org/jira/browse/IGNITE-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16666210#comment-16666210 ]
Pavel Pereslegin edited comment on IGNITE-9858 at 10/27/18 8:00 PM:
--------------------------------------------------------------------
[~agoncharuk], check for client node is performed in {{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}} (I updated ticket description).
We have one instance of IpFinder and following execution order:
1. Server injects ignite instance resource into IpFinder.
{noformat}
at org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:283)
at org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:252)
at org.apache.ignite.internal.processors.resource.GridResourceProcessor.injectGeneric(GridResourceProcessor.java:233)
at org.apache.ignite.internal.managers.GridManagerAdapter.inject(GridManagerAdapter.java:173)
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:257)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
2. Client injects ignite instance resource into IpFinder (same stacktrace).
3. Client initializes SPI, checks for client discovery:
{noformat}
at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113)
at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338)
at org.apache.ignite.spi.discovery.tcp.ClientImpl.spiStart(ClientImpl.java:298)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023)
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
4. Server initializes SPI:
{noformat}
at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113)
at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338)
at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:374)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023)
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
As a result both nodes assume that the local node is a client and don't register local address.
Usually in tests we start client node after starting the server, so this problem is not reproduced.
was (Author: xtern):
[~agoncharuk], check for client node is performed in {{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}} (I updated ticket description).
We have one instance of IpFinder and following execution order:
1. Server injects ignite instance resource into IpFinder.
{noformat}
at org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:283)
at org.apache.ignite.internal.processors.resource.GridResourceProcessor.inject(GridResourceProcessor.java:252)
at org.apache.ignite.internal.processors.resource.GridResourceProcessor.injectGeneric(GridResourceProcessor.java:233)
at org.apache.ignite.internal.managers.GridManagerAdapter.inject(GridManagerAdapter.java:173)
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:257)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
2. Client injects ignite instance resource into IpFinder (same stacktrace).
3. Client initializes SPI, checks for client discovery:
{noformat}
at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113)
at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338)
at org.apache.ignite.spi.discovery.tcp.ClientImpl.spiStart(ClientImpl.java:298)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023)
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
4. Server initializes SPI:
{noformat}
at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.discoveryClientMode(TcpDiscoveryIpFinderAdapter.java:113)
at org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinderAdapter.initializeLocalAddresses(TcpDiscoveryIpFinderAdapter.java:61)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoveryImpl.registerLocalNodeAddress(TcpDiscoveryImpl.java:338)
at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:374)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2023)
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
{noformat}
As a result both nodes assume that the local node is a client and don't register local address.
> [Test Failed] SystemCacheNotConfiguredTest#test flaky fails on TC (timeout).
> ----------------------------------------------------------------------------
>
> Key: IGNITE-9858
> URL: https://issues.apache.org/jira/browse/IGNITE-9858
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.6
> Reporter: Pavel Pereslegin
> Assignee: Pavel Pereslegin
> Priority: Major
> Labels: MakeTeamcityGreenAgain
> Fix For: 2.8
>
>
> SystemCacheNotConfiguredTest hangs sometimes on TeamCity (timeout).
> Example of such failures on master branch: [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&buildTypeId=&tab=testDetails&testNameId=-2762467041583095183&order=TEST_STATUS_DESC&itemsCount=50&branch_IgniteTests24Java8=%3Cdefault%3E]
> When we using ip finder in shared mode each node should register self address (except clients, obviously).
> Check that the node is a client uses installed (via DI @IgniteInstanceResource) Ignite instance (see {{TcpDiscoveryIpFinderAdapter#initializeLocalAddresses}}).
> So when client and server starts simultaneously, the following scenario is possible - Ignite server injected at first, then the Ignite client injected when the SPI is initialized ({{spiStart}}) both nodes assume that the local node is a client and don't register local address.
> {noformat}
> [2018-10-11 18:03:49,794][WARN ][tcp-client-disco-msg-worker-#57%client%][TcpDiscoverySpi] IP finder returned empty addresses list. Please check IP finder configuration. Will retry every 2000 ms. Change 'reconnectDelay' to configure the frequency of retries.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)