[jira] [Created] (SOLR-13036) JettySolrRunner's port retry is broken

["Gus Heck (JIRA)" <ji...@apache.org>]

Gus Heck created SOLR-13036:
-------------------------------

             Summary: JettySolrRunner's port retry is broken
                 Key: SOLR-13036
                 URL: https://issues.apache.org/jira/browse/SOLR-13036
             Project: Solr
          Issue Type: Bug
      Security Level: Public (Default Security Level. Issues are Public)
          Components: Tests
    Affects Versions: master (8.0)
            Reporter: Gus Heck


I was running tests and hit the following stack trace which shows the BindException getting wrapped by an IOException, which then evades our port already bound retry logic in JettySolrRunner#retryOnPortBindFailure() which is catching BindException directly. The result is that we never retry as intended. This observation from reading the code is supported by the fact that the logging statement starting with "Port in use..." did not appear in the log.
{code:java}
build.out4.txt- [beaster] [18:06:13.618] ERROR 25.9s J15 | TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader <<<
build.out4.txt: [beaster] > Throwable #1: java.io.IOException: Failed to bind to /127.0.0.1:36497
build.out4.txt- [beaster] > at __randomizedtesting.SeedInfo.seed([FD0124C33BA78623:83EA04D3F8C08919]:0)
build.out4.txt- [beaster] > at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
build.out4.txt- [beaster] > at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
build.out4.txt- [beaster] > at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
build.out4.txt- [beaster] > at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
build.out4.txt- [beaster] > at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
build.out4.txt- [beaster] > at org.eclipse.jetty.server.Server.doStart(Server.java:394)
build.out4.txt- [beaster] > at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
build.out4.txt- [beaster] > at org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:510)
build.out4.txt- [beaster] > at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:449)
build.out4.txt- [beaster] > at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:417)
build.out4.txt- [beaster] > at org.apache.solr.cloud.TestCloudConsistency.addDocWhenOtherReplicasAreNetworkPartitioned(TestCloudConsistency.java:225)
build.out4.txt- [beaster] > at org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader(TestCloudConsistency.java:136)
build.out4.txt- [beaster] > at org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader(TestCloudConsistency.java:98)
build.out4.txt- [beaster] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
build.out4.txt- [beaster] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
build.out4.txt- [beaster] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
build.out4.txt- [beaster] > at java.lang.reflect.Method.invoke(Method.java:498)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
build.out4.txt- [beaster] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
build.out4.txt- [beaster] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
build.out4.txt- [beaster] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
build.out4.txt- [beaster] > at java.lang.Thread.run(Thread.java:748)
build.out4.txt- [beaster] > Caused by: java.net.BindException: Address already in use
build.out4.txt- [beaster] > at sun.nio.ch.Net.bind0(Native Method)
build.out4.txt- [beaster] > at sun.nio.ch.Net.bind(Net.java:433)
build.out4.txt- [beaster] > at sun.nio.ch.Net.bind(Net.java:425)
build.out4.txt- [beaster] > at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
build.out4.txt- [beaster] > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
build.out4.txt- [beaster] > at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
build.out4.txt- [beaster] > ... 51 more

{code}
SOLR-7339 first noticed this but seems to have not tracked it down because other bigger problems were extant. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org