You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Andrey Davydov <an...@gmail.com> on 2019/07/09 12:04:25 UTC

Cluster crush on 2.7.0

Hello all,

Sometimes we get very strange ignite cluster crush during execution of tests for our system.

2019-07-09 11:02:59,710 [main] ERROR com.imperva.ddc.core.Driver:176 - Ldap Connection to nodelegateddomen.local failed
<Full stack trace at the and of letter>
2019-07-09 11:02:59,715 [main] ERROR com.imperva.ddc.core.Driver:116 - Test connection has failed. Results: Connection to host nodelegateddomen.dev002.local has failed. Reason: com.imperva.ddc.core.exceptions.InvalidConnectionException: org.apache.directory.ldap.client.api.exception.InvalidConnectionException: Cannot connect to the server: Hostname 'nodelegateddomen.local' could not be resolved.
2019-07-09 11:03:05,482 [db-checkpoint-thread-#10414] ERROR :134 - Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.processors.cache.persistence.StorageException: Failed to write checkpoint entry [ptr=FileWALPointer [idx=0, fileOff=2032270, len=31143], cpTs=1562670185474, cpId=1bab48e6-f29c-4fb3-acd0-385783244ad9, type=START]]]
org.apache.ignite.internal.processors.cache.persistence.StorageException: Failed to write checkpoint entry [ptr=FileWALPointer [idx=0, fileOff=2032270, len=31143], cpTs=1562670185474, cpId=1bab48e6-f29c-4fb3-acd0-385783244ad9, type=START]
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.writeCheckpointEntry(GridCacheDatabaseSharedManager.java:2853) ~[ignite-core-2.7.0.jar:2.7.0]
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:3841) ~[ignite-core-2.7.0.jar:2.7.0]
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:3279) [ignite-core-2.7.0.jar:2.7.0]
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3178) [ignite-core-2.7.0.jar:2.7.0]
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) [ignite-core-2.7.0.jar:2.7.0]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
Caused by: java.nio.file.NoSuchFileException: /buildir/testdir/ignitewd/db/storage/node00-98797cd6-5f62-4e0f-8bbd-08163bb111ae/cp/1562670185474-1bab48e6-f29c-4fb3-acd0-385783244ad9-START.bin.tmp
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:1.8.0_212]
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_212]
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_212]
	at sun.nio.fs.UnixFileSystemProvider.newAsynchronousFileChannel(UnixFileSystemProvider.java:196) ~[?:1.8.0_212]
	at java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:248) ~[?:1.8.0_212]
	at java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:301) ~[?:1.8.0_212]
	at org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIO.<init>(AsyncFileIO.java:57) ~[ignite-core-2.7.0.jar:2.7.0]
	at org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory.create(AsyncFileIOFactory.java:53) ~[ignite-core-2.7.0.jar:2.7.0]
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.writeCheckpointEntry(GridCacheDatabaseSharedManager.java:2836) ~[ignite-core-2.7.0.jar:2.7.0]
	... 5 more

I don’t know way to reproduce =(, But we got this crash twice during last month, on the same test. And this test don’t make any payload for cluster, this test is to check how our authorization subsystem handle case when ActiveDirectory is not available. No any manipulation with grid. 

I have no any idea how it works, but both cluster crush was on the same place.

We use Imperva as AD client

                <artifactId>ddc-core</artifactId>
                <groupId>com.imperva.ddc</groupId>
                <version>7.3.3.0.0.0</version>

We use embedded Ignite with enabled percistence.

Thanks for any help.

Andrey.

2019-07-09 11:02:59,710 [main] ERROR com.imperva.ddc.core.Driver:176 - Ldap Connection to nodelegateddomen.local failed
org.apache.directory.ldap.client.api.exception.InvalidConnectionException: Cannot connect to the server: Hostname 'nodelegateddomen.local' could not be resolved.
	at org.apache.directory.ldap.client.api.LdapNetworkConnection.connect(LdapNetworkConnection.java:758) ~[api-all-1.0.3.jar:1.0.3]
	at org.apache.directory.ldap.client.api.LdapNetworkConnection.bindAsync(LdapNetworkConnection.java:1368) ~[api-all-1.0.3.jar:1.0.3]
	at org.apache.directory.ldap.client.api.LdapNetworkConnection.bind(LdapNetworkConnection.java:1293) ~[api-all-1.0.3.jar:1.0.3]
	at com.imperva.ddc.core.Driver.getLdapConnection(Driver.java:148) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.Driver.createConnection(Driver.java:81) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.Driver.doConnect(Driver.java:68) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.Driver.connect(Driver.java:44) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.DriverRobustDecorator.retry(DriverRobustDecorator.java:50) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.DriverRobustDecorator.connect(DriverRobustDecorator.java:29) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.DriverHostResolverDecorator.connect(DriverHostResolverDecorator.java:38) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.QueryRequestNonPagedExecutor.execute(QueryRequestNonPagedExecutor.java:45) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.Executor.execute(Executor.java:26) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.imperva.ddc.core.Connector.execute(Connector.java:106) [ddc-core-7.3.3.0.0.0.jar:?]
	at com.example.utils.activedirectory.ActiveDirectoryApi.resolveDistinguishedName(ActiveDirectoryApi.java:183) [classes/:?]
	at com.example.utils.activedirectory.ActiveDirectoryApi.isEnabled(ActiveDirectoryApi.java:129) [classes/:?]
	at com.example.utils.activedirectory.ActiveDirectoryApiTest.testGetGroups(ActiveDirectoryApiTest.java:160) [test-classes/:?]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) [junit-4.12.jar:4.12]
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12]
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) [junit-4.12.jar:4.12]
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) [junit-4.12.jar:4.12]
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) [junit-4.12.jar:4.12]
	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) [junit-4.12.jar:4.12]
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) [junit-4.12.jar:4.12]
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) [junit-4.12.jar:4.12]
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) [junit-4.12.jar:4.12]
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) [junit-4.12.jar:4.12]
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) [junit-4.12.jar:4.12]
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363) [junit-4.12.jar:4.12]
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at com.example.CustomSpringJUnit4Runner.run(CustomSpringJUnit4Runner.java:27) [test-classes/:?]
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) [surefire-junit4-2.22.1.jar:2.22.1]
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) [surefire-junit4-2.22.1.jar:2.22.1]
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) [surefire-junit4-2.22.1.jar:2.22.1]
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) [surefire-junit4-2.22.1.jar:2.22.1]
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) [surefire-booter-2.22.1.jar:2.22.1]
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) [surefire-booter-2.22.1.jar:2.22.1]
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) [surefire-booter-2.22.1.jar:2.22.1]
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) [surefire-booter-2.22.1.jar:2.22.1]
Caused by: java.nio.channels.UnresolvedAddressException
	at sun.nio.ch.Net.checkAddress(Net.java:101) ~[?:1.8.0_212]
	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) ~[?:1.8.0_212]
	at org.apache.mina.transport.socket.nio.NioSocketConnector.connect(NioSocketConnector.java:187) ~[mina-core-2.0.21.jar:?]
	at org.apache.mina.transport.socket.nio.NioSocketConnector.connect(NioSocketConnector.java:47) ~[mina-core-2.0.21.jar:?]
	at org.apache.mina.core.polling.AbstractPollingIoConnector.connect0(AbstractPollingIoConnector.java:370) ~[mina-core-2.0.21.jar:?]
	at org.apache.mina.core.service.AbstractIoConnector.connect(AbstractIoConnector.java:322) ~[mina-core-2.0.21.jar:?]
	at org.apache.mina.core.service.AbstractIoConnector.connect(AbstractIoConnector.java:206) ~[mina-core-2.0.21.jar:?]
	at org.apache.directory.ldap.client.api.LdapNetworkConnection.connect(LdapNetworkConnection.java:665) ~[api-all-1.0.3.jar:1.0.3]
	... 48 more

Re: Cluster crush on 2.7.0

Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!

I can't comment on your LDAP situation, but you can see errors like this
one in test if you clear Ignite work dir before node is completely stopped.

See for example https://issues.apache.org/jira/browse/IGNITE-8797

Regards,
-- 
Ilya Kasnacheev


вт, 9 июл. 2019 г. в 15:04, Andrey Davydov <an...@gmail.com>:

>
>
> Hello all,
>
>
>
> Sometimes we get very strange ignite cluster crush during execution of
> tests for our system.
>
>
>
> 2019-07-09 11:02:59,710 [main] ERROR com.imperva.ddc.core.Driver:176 -
> Ldap Connection to nodelegateddomen.local failed
>
> <Full stack trace at the and of letter>
>
> 2019-07-09 11:02:59,715 [main] ERROR com.imperva.ddc.core.Driver:116 -
> Test connection has failed. Results: Connection to host
> nodelegateddomen.dev002.local has failed. Reason:
> com.imperva.ddc.core.exceptions.InvalidConnectionException:
> org.apache.directory.ldap.client.api.exception.InvalidConnectionException:
> Cannot connect to the server: Hostname 'nodelegateddomen.local' could not
> be resolved.
>
> 2019-07-09 11:03:05,482 [db-checkpoint-thread-#10414] ERROR :134 -
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler
> [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=class
> o.a.i.i.processors.cache.persistence.StorageException: Failed to write
> checkpoint entry [ptr=FileWALPointer [idx=0, fileOff=2032270, len=31143],
> cpTs=1562670185474, cpId=1bab48e6-f29c-4fb3-acd0-385783244ad9, type=START]]]
>
> org.apache.ignite.internal.processors.cache.persistence.StorageException:
> Failed to write checkpoint entry [ptr=FileWALPointer [idx=0,
> fileOff=2032270, len=31143], cpTs=1562670185474,
> cpId=1bab48e6-f29c-4fb3-acd0-385783244ad9, type=START]
>
>              at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.writeCheckpointEntry(GridCacheDatabaseSharedManager.java:2853)
> ~[ignite-core-2.7.0.jar:2.7.0]
>
>              at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:3841)
> ~[ignite-core-2.7.0.jar:2.7.0]
>
>              at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:3279)
> [ignite-core-2.7.0.jar:2.7.0]
>
>              at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3178)
> [ignite-core-2.7.0.jar:2.7.0]
>
>              at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> [ignite-core-2.7.0.jar:2.7.0]
>
>              at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
>
> Caused by: java.nio.file.NoSuchFileException:
> /buildir/testdir/ignitewd/db/storage/node00-98797cd6-5f62-4e0f-8bbd-08163bb111ae/cp/1562670185474-1bab48e6-f29c-4fb3-acd0-385783244ad9-START.bin.tmp
>
>              at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> ~[?:1.8.0_212]
>
>              at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> ~[?:1.8.0_212]
>
>              at
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> ~[?:1.8.0_212]
>
>              at
> sun.nio.fs.UnixFileSystemProvider.newAsynchronousFileChannel(UnixFileSystemProvider.java:196)
> ~[?:1.8.0_212]
>
>              at
> java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:248)
> ~[?:1.8.0_212]
>
>              at
> java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:301)
> ~[?:1.8.0_212]
>
>              at
> org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIO.<init>(AsyncFileIO.java:57)
> ~[ignite-core-2.7.0.jar:2.7.0]
>
>              at
> org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory.create(AsyncFileIOFactory.java:53)
> ~[ignite-core-2.7.0.jar:2.7.0]
>
>              at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.writeCheckpointEntry(GridCacheDatabaseSharedManager.java:2836)
> ~[ignite-core-2.7.0.jar:2.7.0]
>
>              ... 5 more
>
>
>
> I don’t know way to reproduce =(, But we got this crash twice during last
> month, on the same test. And this test don’t make any payload for cluster,
> this test is to check how our authorization subsystem handle case when
> ActiveDirectory is not available. No any manipulation with grid.
>
>
>
> I have no any idea how it works, but both cluster crush was on the same
> place.
>
>
>
> We use Imperva as AD client
>
>
>
>                 <artifactId>ddc-core</artifactId>
>
>                 <groupId>com.imperva.ddc</groupId>
>
>                 <version>7.3.3.0.0.0</version>
>
>
>
> We use embedded Ignite with enabled percistence.
>
>
>
> Thanks for any help.
>
>
>
> Andrey.
>
>
>
> 2019-07-09 11:02:59,710 [main] ERROR com.imperva.ddc.core.Driver:176 -
> Ldap Connection to nodelegateddomen.local failed
>
> org.apache.directory.ldap.client.api.exception.InvalidConnectionException:
> Cannot connect to the server: Hostname 'nodelegateddomen.local' could not
> be resolved.
>
>              at
> org.apache.directory.ldap.client.api.LdapNetworkConnection.connect(LdapNetworkConnection.java:758)
> ~[api-all-1.0.3.jar:1.0.3]
>
>              at
> org.apache.directory.ldap.client.api.LdapNetworkConnection.bindAsync(LdapNetworkConnection.java:1368)
> ~[api-all-1.0.3.jar:1.0.3]
>
>              at
> org.apache.directory.ldap.client.api.LdapNetworkConnection.bind(LdapNetworkConnection.java:1293)
> ~[api-all-1.0.3.jar:1.0.3]
>
>              at
> com.imperva.ddc.core.Driver.getLdapConnection(Driver.java:148)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at
> com.imperva.ddc.core.Driver.createConnection(Driver.java:81)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at com.imperva.ddc.core.Driver.doConnect(Driver.java:68)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at com.imperva.ddc.core.Driver.connect(Driver.java:44)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at
> com.imperva.ddc.core.DriverRobustDecorator.retry(DriverRobustDecorator.java:50)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at
> com.imperva.ddc.core.DriverRobustDecorator.connect(DriverRobustDecorator.java:29)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at
> com.imperva.ddc.core.DriverHostResolverDecorator.connect(DriverHostResolverDecorator.java:38)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at
> com.imperva.ddc.core.QueryRequestNonPagedExecutor.execute(QueryRequestNonPagedExecutor.java:45)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at com.imperva.ddc.core.Executor.execute(Executor.java:26)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at com.imperva.ddc.core.Connector.execute(Connector.java:106)
> [ddc-core-7.3.3.0.0.0.jar:?]
>
>              at com.example.utils.activedirectory.ActiveDirectoryApi.resolveDistinguishedName(ActiveDirectoryApi.java:183)
> [classes/:?]
>
>              at com.example.utils.activedirectory.ActiveDirectoryApi.isEnabled(ActiveDirectoryApi.java:129)
> [classes/:?]
>
>              at com.example.utils.activedirectory.ActiveDirectoryApiTest.testGetGroups(ActiveDirectoryApiTest.java:160)
> [test-classes/:?]
>
>              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) ~[?:1.8.0_212]
>
>              at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_212]
>
>              at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_212]
>
>              at java.lang.reflect.Method.invoke(Method.java:498)
> ~[?:1.8.0_212]
>
>              at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> [junit-4.12.jar:4.12]
>
>              at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> [junit-4.12.jar:4.12]
>
>              at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> [junit-4.12.jar:4.12]
>
>              at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> [junit-4.12.jar:4.12]
>
>              at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> [junit-4.12.jar:4.12]
>
>              at
> org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
> [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
>
>              at
> org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
> [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
>
>              at
> org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
> [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
>
>              at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> [junit-4.12.jar:4.12]
>
>              at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252)
> [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
>
>              at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
> [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
>
>              at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> [junit-4.12.jar:4.12]
>
>              at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> [junit-4.12.jar:4.12]
>
>              at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> [junit-4.12.jar:4.12]
>
>              at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> [junit-4.12.jar:4.12]
>
>              at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> [junit-4.12.jar:4.12]
>
>              at
> org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
> [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
>
>              at
> org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
> [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
>
>              at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> [junit-4.12.jar:4.12]
>
>              at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
> [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
>
>              at com.example.CustomSpringJUnit4Runner.run(CustomSpringJUnit4Runner.java:27)
> [test-classes/:?]
>
>              at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> [surefire-junit4-2.22.1.jar:2.22.1]
>
>              at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> [surefire-junit4-2.22.1.jar:2.22.1]
>
>              at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> [surefire-junit4-2.22.1.jar:2.22.1]
>
>              at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> [surefire-junit4-2.22.1.jar:2.22.1]
>
>              at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> [surefire-booter-2.22.1.jar:2.22.1]
>
>              at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> [surefire-booter-2.22.1.jar:2.22.1]
>
>              at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> [surefire-booter-2.22.1.jar:2.22.1]
>
>              at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> [surefire-booter-2.22.1.jar:2.22.1]
>
> Caused by: java.nio.channels.UnresolvedAddressException
>
>              at sun.nio.ch.Net.checkAddress(Net.java:101) ~[?:1.8.0_212]
>
>              at
> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
> ~[?:1.8.0_212]
>
>              at
> org.apache.mina.transport.socket.nio.NioSocketConnector.connect(NioSocketConnector.java:187)
> ~[mina-core-2.0.21.jar:?]
>
>              at
> org.apache.mina.transport.socket.nio.NioSocketConnector.connect(NioSocketConnector.java:47)
> ~[mina-core-2.0.21.jar:?]
>
>              at
> org.apache.mina.core.polling.AbstractPollingIoConnector.connect0(AbstractPollingIoConnector.java:370)
> ~[mina-core-2.0.21.jar:?]
>
>              at
> org.apache.mina.core.service.AbstractIoConnector.connect(AbstractIoConnector.java:322)
> ~[mina-core-2.0.21.jar:?]
>
>              at
> org.apache.mina.core.service.AbstractIoConnector.connect(AbstractIoConnector.java:206)
> ~[mina-core-2.0.21.jar:?]
>
>              at
> org.apache.directory.ldap.client.api.LdapNetworkConnection.connect(LdapNetworkConnection.java:665)
> ~[api-all-1.0.3.jar:1.0.3]
>
>              ... 48 more
>