You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/07/27 00:39:20 UTC

[JENKINS] Lucene-Solr-Tests-7.x - Build # 105 - Still unstable

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/105/

1 tests failed.
FAILED:  org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelExecutorStream

Error Message:
Error from server at https://127.0.0.1:43383/solr/mainCorpus_shard2_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/mainCorpus_shard2_replica_n2/update. Reason: <pre>    Can not find: /solr/mainCorpus_shard2_replica_n2/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:43383/solr/mainCorpus_shard2_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/mainCorpus_shard2_replica_n2/update. Reason:
<pre>    Can not find: /solr/mainCorpus_shard2_replica_n2/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([F1CB33766526C0B0:4CDC466F5C0AFDED]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelExecutorStream(StreamExpressionTest.java:6844)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:43383/solr/mainCorpus_shard2_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/mainCorpus_shard2_replica_n2/update. Reason:
<pre>    Can not find: /solr/mainCorpus_shard2_replica_n2/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:588)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more




Build Log:
[...truncated 18 lines...]
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git://git.apache.org/lucene-solr.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:812)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1079)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1110)
	at hudson.scm.SCM.checkout(SCM.java:495)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1276)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:560)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:485)
	at hudson.model.Run.execute(Run.java:1735)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:405)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: fatal: unable to connect to git.apache.org:
git.apache.org[0: 54.84.58.65]: errno=Connection refused


	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1903)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1622)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71)
	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
	at hudson.remoting.UserRequest.perform(UserRequest.java:153)
	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
	at hudson.remoting.Request$2.run(Request.java:336)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
	at ......remote call to lucene(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
	at hudson.remoting.Channel.call(Channel.java:830)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
	at sun.reflect.GeneratedMethodAccessor864.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
	at com.sun.proxy.$Proxy104.execute(Unknown Source)
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:810)
	... 11 more
ERROR: Error fetching remote repo 'origin'
Retrying after 10 seconds
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git://git.apache.org/lucene-solr.git # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
Fetching upstream changes from git://git.apache.org/lucene-solr.git
 > git --version # timeout=10
 > git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/branch_7x^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/branch_7x^{commit} # timeout=10
Checking out Revision cef31d62411b6377fdb517ad2d8030c629a1da18 (refs/remotes/origin/branch_7x)
Commit message: "SOLR-10760: fix another Solrj test"
 > git config core.sparsecheckout # timeout=10
 > git checkout -f cef31d62411b6377fdb517ad2d8030c629a1da18
 > git rev-list e9cf285baf88628a994c2f2a9b1a8d54867b636c # timeout=10
No emails were triggered.
[Lucene-Solr-Tests-7.x] $ /home/jenkins/tools/ant/apache-ant-1.8.4/bin/ant jenkins-hourly
Buildfile: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml

jenkins-hourly:

-print-java-info:
[java-info] java version "1.8.0_131"
[java-info] Java(TM) SE Runtime Environment (1.8.0_131-b11, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (25.131-b11, Oracle Corporation)
[java-info] Test args: []

clean:

clean:

clean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve-groovy:
[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.8 in public
[ivy:cachepath] :: resolution report :: resolve 129ms :: artifacts dl 3ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |
	---------------------------------------------------------------------

-test-with-heapdumps-enabled:
     [echo] Java HotSpot(TM) 64-Bit Server VM: Enabling heap dumps on OutOfMemoryError to dir '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/heapdumps'.
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/heapdumps

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve-groovy:

test:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

-clover.load:

resolve-groovy:

-init-totals:

test-core:

-clover.disable:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

-clover.load:

-clover.classpath:

-clover.setup:

clover:

-check-git-state:

-git-cleanroot:

-copy-git-state:
     [copy] Copying 1 file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build

git-autoclean:

resolve:

init:

compile-core:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java
    [javac] Compiling 823 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/util/graph/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/document/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/util/bkd/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/codecs/compressing/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene60/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/codecs/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/util/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/index/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/util/fst/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/analysis/tokenattributes/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/search/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene62/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/util/packed/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/util/mutable/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene70/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/codecs/perfield/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/analysis/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/analysis/standard/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/store/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/codecs/blocktree/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/search/spans/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/search/similarities/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/util/automaton/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene50/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java/org/apache/lucene/geo/package-info.class
     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/java

compile-test-framework:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

init:

compile-lucene-core:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

init:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

compile-core:

compile-codecs:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

common.init:

compile-lucene-core:

init:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

compile-core:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/codecs/classes/java
    [javac] Compiling 67 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/codecs/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/bloom/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/memory/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/simpletext/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blockterms/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blocktreeords/package-info.class
     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/codecs/classes/java

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

common.compile-core:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java
    [javac] Compiling 188 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/asserting/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/ramonly/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java/org/apache/lucene/mockfile/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cheapbastard/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/mockrandom/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cranky/package-info.class
    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/compressing/dummy/package-info.class
     [copy] Copying 4 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/test-framework/classes/java

compile-core:

compile-test:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/test
    [javac] Compiling 461 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/test
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/classes/test

install-junit4-taskdef:

validate:

resolve-groovy:

-init-totals:

-test:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/test
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/core/test/temp
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/.caches/test-stats/core
   [junit4] <JUnit4> says שלום! Master seed: 7F087683BEFA2C84
   [junit4] Executing 454 suites with 3 JVMs.
   [junit4] 
   [junit4] Started J0 PID(16518@localhost).
   [junit4] Started J2 PID(16520@localhost).
   [junit4] Started J1 PID(16519@localhost).
   [junit4] Suite: org.apache.lucene.util.TestDocIdSetBuilder
   [junit4] Completed [2/454] on J0 in 1.09s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocCount
   [junit4] Completed [3/454] on J0 in 1.25s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSortRandom
   [junit4] Completed [4/454] on J1 in 3.46s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery
   [junit4] Completed [5/454] on J2 in 3.52s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCharsRefBuilder
   [junit4] Completed [6/454] on J2 in 0.08s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFilterLeafReader
   [junit4] Completed [7/454] on J1 in 0.42s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestVirtualMethod
   [junit4] Completed [8/454] on J1 in 0.09s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestField
   [junit4] Completed [9/454] on J2 in 0.35s, 23 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNeverDelete
   [junit4] Completed [10/454] on J0 in 2.30s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.bkd.TestMutablePointsReaderUtils
   [junit4] Completed [11/454] on J2 in 1.65s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDateSort
   [junit4] Completed [12/454] on J2 in 0.37s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanBoostQuery
   [junit4] Completed [13/454] on J2 in 0.07s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNoMergePolicy
   [junit4] Completed [14/454] on J2 in 0.41s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestCharTermAttributeImpl
   [junit4] Completed [15/454] on J2 in 0.83s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexManyDocuments
   [junit4] Completed [16/454] on J2 in 2.80s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCachingTokenFilter
   [junit4] Completed [17/454] on J2 in 0.20s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestSearchForDuplicates
   [junit4] Completed [18/454] on J2 in 0.53s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAddIndexes
   [junit4] Completed [19/454] on J1 in 8.66s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRecyclingByteBlockAllocator
   [junit4] Completed [20/454] on J1 in 0.11s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanScorer
   [junit4] Completed [21/454] on J0 in 9.47s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminism
   [junit4] Completed [22/454] on J1 in 2.06s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.TestCodecUtil
   [junit4] Completed [23/454] on J1 in 0.27s, 16 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelReaderEmptyIndex
   [junit4] Completed [24/454] on J1 in 0.22s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCheckIndex
   [junit4] Completed [25/454] on J1 in 2.65s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocValuesRewriteMethod
   [junit4] Completed [26/454] on J2 in 6.98s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMatchNoDocsQuery
   [junit4] Completed [27/454] on J1 in 0.15s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanations
   [junit4] Completed [28/454] on J0 in 5.76s, 70 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNoDeletionPolicy
   [junit4] Completed [29/454] on J0 in 0.11s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiFields
   [junit4] Completed [30/454] on J2 in 2.92s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpansEnum
   [junit4] Completed [31/454] on J0 in 0.73s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntsRef
   [junit4] Completed [32/454] on J2 in 0.07s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsOfNonMatches
   [junit4] Completed [33/454] on J2 in 0.64s, 70 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestUnicodeUtil
   [junit4] Completed [34/454] on J0 in 1.71s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSort
   [junit4] Completed [35/454] on J0 in 0.70s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestNamedSPILoader
   [junit4] Completed [36/454] on J0 in 0.04s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntArrayDocIdSet
   [junit4] Completed [37/454] on J2 in 2.51s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTwoPhaseCommitTool
   [junit4] Completed [38/454] on J2 in 0.06s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.bkd.Test2BBKDPoints
   [junit4] IGNOR/A 0.01s J2 | Test2BBKDPoints.test1D
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))
   [junit4] IGNOR/A 0.00s J2 | Test2BBKDPoints.test2D
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))
   [junit4] Completed [39/454] on J2 in 0.03s, 2 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestIndexSearcher
   [junit4] Completed [40/454] on J2 in 0.72s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestVersion
   [junit4] Completed [41/454] on J2 in 0.11s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpQuery
   [junit4] Completed [42/454] on J2 in 0.17s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPersistentSnapshotDeletionPolicy
   [junit4] Completed [43/454] on J0 in 2.04s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermVectors
   [junit4] Completed [44/454] on J2 in 1.14s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
   [junit4] Completed [45/454] on J0 in 4.85s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.fst.TestFSTs
   [junit4] IGNOR/A 0.01s J1 | TestFSTs.testBigSet
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [46/454] on J1 in 14.21s, 20 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestInfoStream
   [junit4] Completed [47/454] on J1 in 0.10s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestMultiMMap
   [junit4] Completed [48/454] on J2 in 7.72s, 54 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestForUtil
   [junit4] Completed [49/454] on J2 in 0.16s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveCodecHeader
   [junit4] Completed [50/454] on J2 in 1.69s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSameTokenSamePosition
   [junit4] Completed [51/454] on J2 in 0.10s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBinaryDocValuesUpdates
   [junit4] IGNOR/A 0.01s J0 | TestBinaryDocValuesUpdates.testTonsOfUpdates
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [52/454] on J0 in 8.06s, 29 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPerSegmentDeletes
   [junit4] Completed [53/454] on J0 in 0.32s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterConfig
   [junit4] Completed [54/454] on J0 in 0.16s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestReqExclBulkScorer
   [junit4] Completed [55/454] on J0 in 0.10s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery2
   [junit4] Completed [56/454] on J2 in 3.08s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestLogMergePolicy
   [junit4] Completed [57/454] on J2 in 0.10s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterUnicode
   [junit4] Completed [58/454] on J1 in 7.43s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFixedLengthBytesRefArray
   [junit4] Completed [59/454] on J2 in 0.31s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopDocsCollector
   [junit4] Completed [60/454] on J2 in 0.14s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestByteArrayDataInput
   [junit4] Completed [61/454] on J1 in 0.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermsEnum2
   [junit4] Completed [62/454] on J1 in 2.13s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestNumericUtils
   [junit4] Completed [63/454] on J2 in 6.28s, 24 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestByteSlices
   [junit4] Completed [64/454] on J1 in 5.56s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntroSorter
   [junit4] Completed [65/454] on J1 in 0.47s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPositiveScoresOnlyCollector
   [junit4] Completed [66/454] on J1 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTransactionRollback
   [junit4] Completed [67/454] on J1 in 0.50s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocInverterPerFieldErrorInfo
   [junit4] Completed [68/454] on J1 in 0.14s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestExternalCodecs
   [junit4] Completed [69/454] on J1 in 1.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPrefixCodedTerms
   [junit4] Completed [70/454] on J1 in 0.43s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestBasics
   [junit4] Completed [71/454] on J2 in 6.06s, 26 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestRollingUpdates
   [junit4] Completed [72/454] on J2 in 3.16s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMixedDocValuesUpdates
   [junit4] IGNOR/A 0.00s J1 | TestMixedDocValuesUpdates.testTonsOfUpdates
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [73/454] on J1 in 6.00s, 4 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestBytesRefAttImpl
   [junit4] Completed [74/454] on J1 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestFilterSpans
   [junit4] Completed [75/454] on J1 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.bkd.TestDocIdsWriter
   [junit4] Completed [76/454] on J1 in 7.89s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestConjunctionDISI
   [junit4] Completed [77/454] on J2 in 11.04s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSubScorerFreqs
   [junit4] Completed [78/454] on J2 in 0.18s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull
   [junit4] Completed [79/454] on J2 in 1.03s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTermRangeQuery
   [junit4] Completed [80/454] on J2 in 0.27s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanTermQuery
   [junit4] Completed [81/454] on J2 in 0.04s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBoolean2
   [junit4] Completed [82/454] on J0 in 30.66s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressNRT
   [junit4] Completed [83/454] on J1 in 5.59s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermdocPerf
   [junit4] Completed [84/454] on J1 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
   [junit4] IGNOR/A 0.00s J1 | TestDuelingCodecsAtNight.testEquals
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J1 | TestDuelingCodecsAtNight.testCrazyReaderEquals
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J1 | TestDuelingCodecsAtNight.testBigEquals
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [85/454] on J1 in 0.02s, 3 tests, 3 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleSearchEquivalence
   [junit4] Completed [86/454] on J0 in 0.90s, 17 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestInPlaceMergeSorter
   [junit4] Completed [87/454] on J1 in 0.53s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSameScoresWithThreads
   [junit4] Completed [88/454] on J1 in 1.19s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BPostingsBytes
   [junit4] IGNOR/A 0.00s J1 | Test2BPostingsBytes.test
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~20GB-30GB of space and 10 minutes))
   [junit4] Completed [89/454] on J1 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSloppyMath
   [junit4] Completed [90/454] on J2 in 4.02s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesCheckIndexHeader
   [junit4] Completed [91/454] on J0 in 1.27s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCharacterUtils
   [junit4] Completed [92/454] on J2 in 0.05s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSetOnce
   [junit4] Completed [93/454] on J2 in 0.22s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestDirectory
   [junit4] IGNOR/A 0.01s J1 | TestDirectory.testListAll
   [junit4]    > Assumption #1: this test does not expect extra files
   [junit4] Completed [94/454] on J1 in 0.64s, 4 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTryDelete
   [junit4] Completed [95/454] on J2 in 0.25s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BPositions
   [junit4] IGNOR/A 0.00s J2 | Test2BPositions.test
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=uses lots of space and takes a few minutes))
   [junit4] Completed [96/454] on J2 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFilterDirectoryReader
   [junit4] Completed [97/454] on J1 in 0.22s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.Test2BPagedBytes
   [junit4] IGNOR/A 0.00s J2 | Test2BPagedBytes.test
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=You must increase heap to > 2 G to run this))
   [junit4] Completed [98/454] on J2 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanNearQuery
   [junit4] Completed [99/454] on J2 in 0.05s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDeleteByQuery
   [junit4] Completed [100/454] on J2 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestReadOnlyIndex
   [junit4] Completed [101/454] on J2 in 0.15s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixRandom
   [junit4] Completed [102/454] on J1 in 1.25s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSortRescorer
   [junit4] Completed [103/454] on J2 in 2.22s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene60FieldInfoFormat
   [junit4] Completed [104/454] on J1 in 1.27s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFilterWeight
   [junit4] Completed [105/454] on J2 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestAssertions
   [junit4] Completed [106/454] on J2 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFieldReuse
   [junit4] Completed [107/454] on J2 in 0.06s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMergePolicyWrapper
   [junit4] Completed [108/454] on J2 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSentinelIntSet
   [junit4] Completed [109/454] on J2 in 0.34s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestMathUtil
   [junit4] Completed [110/454] on J2 in 0.03s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestFileSwitchDirectory
   [junit4] IGNOR/A 0.01s J1 | TestFileSwitchDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] IGNOR/A 0.02s J1 | TestFileSwitchDirectory.testPendingDeletions
   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory
   [junit4] Completed [111/454] on J1 in 4.19s, 43 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentInfos
   [junit4] Completed [112/454] on J1 in 0.04s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter
   [junit4] Completed [113/454] on J1 in 1.34s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnJRECrash
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterOnJRECrash.testNRTThreads
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [114/454] on J1 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestWildcard
   [junit4] Completed [115/454] on J1 in 0.22s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestHugeRamFile
   [junit4] Completed [116/454] on J2 in 7.76s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiset
   [junit4] Completed [117/454] on J2 in 0.03s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestAttributeSource
   [junit4] Completed [118/454] on J2 in 0.07s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastDecompressionMode
   [junit4] Completed [119/454] on J1 in 3.79s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat
   [junit4] IGNOR/A 0.00s J0 | TestPerFieldPostingsFormat.testMergeStability
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] IGNOR/A 0.00s J0 | TestPerFieldPostingsFormat.testPostingsEnumReuse
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] Completed [120/454] on J0 in 14.45s, 25 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSparseFixedBitSet
   [junit4] Completed [121/454] on J1 in 3.06s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPositionIncrement
   [junit4] Completed [122/454] on J1 in 0.12s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelTermEnum
   [junit4] Completed [123/454] on J1 in 0.17s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSPIClassIterator
   [junit4] Completed [124/454] on J1 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanations
   [junit4] Completed [125/454] on J1 in 1.71s, 30 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions
   [junit4] IGNOR/A 0.00s J0 | TestIndexWriterExceptions.testTooManyTokens
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [126/454] on J0 in 12.19s, 34 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestFieldMaskingSpanQuery
   [junit4] IGNOR/A 0.01s J0 | TestFieldMaskingSpanQuery.testSpans2
   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723
   [junit4] IGNOR/A 0.00s J0 | TestFieldMaskingSpanQuery.testSimple2
   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723
   [junit4] Completed [127/454] on J0 in 0.31s, 11 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitNorms
   [junit4] Completed [128/454] on J0 in 1.23s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.packed.TestPackedInts
   [junit4] IGNOR/A 0.00s J2 | TestPackedInts.testBlockReaderOverflow
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [129/454] on J2 in 17.99s, 28 tests, 3 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastCompressionMode
   [junit4] Completed [130/454] on J2 in 2.14s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs
   [junit4] IGNOR/A 0.01s J1 | TestSimpleExplanationsWithFillerDocs.testMA2
   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null
   [junit4] IGNOR/A 0.00s J1 | TestSimpleExplanationsWithFillerDocs.testMA1
   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null
   [junit4] Completed [131/454] on J1 in 13.96s, 70 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat3
   [junit4] Completed [132/454] on J1 in 2.15s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDoc
   [junit4] Completed [133/454] on J1 in 0.36s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestAutomatonQueryUnicode
   [junit4] Completed [134/454] on J1 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestSingleInstanceLockFactory
   [junit4] Completed [135/454] on J1 in 2.31s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestNativeFSLockFactory
   [junit4] Completed [136/454] on J0 in 10.12s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNeedsScores
   [junit4] Completed [137/454] on J0 in 0.17s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper
   [junit4] Completed [138/454] on J0 in 0.51s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormatHighCompression
   [junit4] IGNOR/A 0.00s J2 | TestLucene50StoredFieldsFormatHighCompression.testBigDocuments
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [139/454] on J2 in 11.20s, 19 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestQueryRescorer
   [junit4] Completed [140/454] on J2 in 1.02s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopDocsMerge
   [junit4] Completed [141/454] on J2 in 1.73s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpans
   [junit4] Completed [142/454] on J2 in 1.10s, 26 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestNIOFSDirectory
   [junit4] Completed [143/454] on J2 in 3.03s, 42 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BBinaryDocValues
   [junit4] IGNOR/A 0.00s J2 | Test2BBinaryDocValues.testFixedBinary
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))
   [junit4] IGNOR/A 0.00s J2 | Test2BBinaryDocValues.testVariableBinary
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))
   [junit4] Completed [144/454] on J2 in 0.02s, 2 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPointQueries
   [junit4] IGNOR/A 0.00s J1 | TestPointQueries.testRandomBinaryBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J1 | TestPointQueries.testRandomLongsBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [145/454] on J1 in 13.69s, 49 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTermQuery
   [junit4] Completed [146/454] on J1 in 0.07s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIOUtils
   [junit4] Completed [147/454] on J2 in 0.21s, 18 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestScoreCachingWrappingScorer
   [junit4] Completed [148/454] on J1 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestBufferedChecksum
   [junit4] Completed [149/454] on J1 in 0.10s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPhrasePrefixQuery
   [junit4] Completed [150/454] on J1 in 0.09s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestGrowableByteArrayDataOutput
   [junit4] Completed [151/454] on J0 in 11.99s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNot
   [junit4] Completed [152/454] on J0 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTerms
   [junit4] Completed [153/454] on J0 in 0.50s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanRewrites
   [junit4] Completed [154/454] on J0 in 0.72s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestExitableDirectoryReader
   [junit4] Completed [155/454] on J0 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPhraseQuery
   [junit4] Completed [156/454] on J1 in 2.29s, 21 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.mutable.TestMutableValues
   [junit4] Completed [157/454] on J1 in 0.04s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.fst.TestBytesStore
   [junit4] Completed [158/454] on J0 in 3.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanCollection
   [junit4] Completed [159/454] on J0 in 0.10s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestReaderWrapperDVTypeCheck
   [junit4] Completed [160/454] on J0 in 0.11s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocsWithFieldSet
   [junit4] Completed [161/454] on J0 in 0.15s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntroSelector
   [junit4] Completed [162/454] on J0 in 1.70s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNewestSegment
   [junit4] Completed [163/454] on J0 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestForceMergeForever
   [junit4] Completed [164/454] on J0 in 0.95s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy
   [junit4] Completed [165/454] on J0 in 4.33s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene60.TestLucene60PointsFormat
   [junit4] IGNOR/A 0.00s J2 | TestLucene60PointsFormat.testMergeStability
   [junit4]    > Assumption #1: merge is not stable
   [junit4] IGNOR/A 0.00s J2 | TestLucene60PointsFormat.testRandomBinaryBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [166/454] on J2 in 12.46s, 20 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestUniqueTermCount
   [junit4] Completed [167/454] on J0 in 0.12s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestThreadedForceMerge
   [junit4] Completed [168/454] on J0 in 0.70s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestSearch
   [junit4] Completed [169/454] on J0 in 0.03s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNGramPhraseQuery
   [junit4] Completed [170/454] on J0 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFrequencyTrackingRingBuffer
   [junit4] Completed [171/454] on J0 in 0.38s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMergePolicy
   [junit4] Completed [172/454] on J2 in 5.43s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.geo.TestPolygon2D
   [junit4] Completed [173/454] on J2 in 13.00s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.standard.TestStandardAnalyzer
   [junit4] Completed [174/454] on J0 in 20.26s, 36 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecs
   [junit4] Completed [175/454] on J1 in 31.23s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestRateLimiter
   [junit4] IGNOR/A 0.61s J1 | TestRateLimiter.testThreads
   [junit4]    > Assumption #1: actualMBPerSec=17.810428746647563 targetMBPerSec=19.944870684182554
   [junit4] Completed [176/454] on J1 in 0.69s, 2 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestLiveFieldValues
   [junit4] Completed [177/454] on J0 in 1.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestSleepingLockWrapper
   [junit4] Completed [178/454] on J2 in 5.48s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPostingsOffsets
   [junit4] Completed [179/454] on J0 in 2.64s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiPhraseQuery
   [junit4] Completed [180/454] on J0 in 0.34s, 16 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestMSBRadixSorter
   [junit4] Completed [181/454] on J1 in 5.49s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelLeafReader
   [junit4] Completed [182/454] on J1 in 0.43s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMaxDocs
   [junit4] IGNOR/A 0.00s J2 | TestIndexWriterMaxDocs.testExactlyAtTrueLimit
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes over two hours))
   [junit4] Completed [183/454] on J2 in 5.03s, 21 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestCustomSearcherSort
   [junit4] Completed [184/454] on J1 in 2.15s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixQuery
   [junit4] Completed [185/454] on J2 in 6.85s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestStringHelper
   [junit4] Completed [186/454] on J2 in 0.08s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterThreadsToSegments
   [junit4] Completed [187/454] on J2 in 2.90s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.MultiCollectorTest
   [junit4] Completed [188/454] on J2 in 0.10s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestPackedTokenAttributeImpl
   [junit4] Completed [189/454] on J2 in 0.03s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
   [junit4] Completed [190/454] on J1 in 10.06s, 20 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestTimSorter
   [junit4] Completed [191/454] on J1 in 0.32s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocValuesQueries
   [junit4] Completed [192/454] on J1 in 8.60s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestReaderClosed
   [junit4] Completed [193/454] on J1 in 0.04s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestForTooMuchCloning
   [junit4] Completed [194/454] on J1 in 0.16s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestToken
   [junit4] Completed [195/454] on J1 in 0.06s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriter
   [junit4] Completed [196/454] on J2 in 10.60s, 72 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDoubleValuesSource
   [junit4] Completed [197/454] on J1 in 4.10s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestComplexExplanations
   [junit4] Completed [198/454] on J1 in 1.65s, 21 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestGraphTokenizers
   [junit4] Completed [199/454] on J1 in 9.44s, 23 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOutOfFileDescriptors
   [junit4] Completed [200/454] on J1 in 3.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAtomicUpdate
   [junit4] Completed [201/454] on J1 in 3.67s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterReader.testDuringAddIndexes
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [202/454] on J1 in 6.53s, 23 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging
   [junit4] Completed [203/454] on J1 in 9.98s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestShardSearching
   [junit4] Completed [204/454] on J1 in 9.91s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSearcherManager
   [junit4] Completed [205/454] on J1 in 9.19s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanOr
   [junit4] Completed [206/454] on J1 in 2.23s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy
   [junit4] Completed [207/454] on J1 in 3.90s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery
   [junit4] Completed [208/454] on J1 in 1.57s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat
   [junit4] Completed [209/454] on J2 in 66.14s, 110 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers
   [junit4] Completed [210/454] on J0 in 94.81s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressDeletes
   [junit4] Completed [211/454] on J0 in 0.75s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom
   [junit4] Completed [212/454] on J0 in 0.24s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNRTThreads
   [junit4] Completed [213/454] on J2 in 7.00s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressAdvance
   [junit4] Completed [214/454] on J2 in 2.78s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBagOfPositions
   [junit4] Completed [215/454] on J0 in 5.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopFieldCollector
   [junit4] Completed [216/454] on J0 in 0.20s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test4GBStoredFields
   [junit4] IGNOR/A 0.00s J0 | Test4GBStoredFields.test
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [217/454] on J0 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCharFilter
   [junit4] Completed [218/454] on J0 in 0.02s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNorms
   [junit4] Completed [219/454] on J2 in 0.84s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterCommit
   [junit4] Completed [220/454] on J2 in 2.05s, 14 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDelete
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterDelete.testApplyDeletesOnFlush
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [221/454] on J1 in 15.71s, 25 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDirectoryReaderReopen
   [junit4] Completed [222/454] on J2 in 2.72s, 14 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTimeLimitingCollector
   [junit4] Completed [223/454] on J1 in 5.17s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSearchWithThreads
   [junit4] Completed [224/454] on J2 in 4.73s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSearchAfter
   [junit4] Completed [225/454] on J1 in 0.85s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestHighCompressionMode
   [junit4] Completed [226/454] on J1 in 2.73s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler
   [junit4] Completed [227/454] on J2 in 6.72s, 16 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestLongPostings
   [junit4] Completed [228/454] on J2 in 1.78s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterStallControl
   [junit4] Completed [229/454] on J1 in 7.60s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressIndexing
   [junit4] Completed [230/454] on J2 in 3.00s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSnapshotDeletionPolicy
   [junit4] Completed [231/454] on J2 in 1.21s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize
   [junit4] Completed [232/454] on J1 in 3.45s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads
   [junit4] Completed [233/454] on J2 in 1.46s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50TermVectorsFormat
   [junit4] Completed [234/454] on J0 in 28.04s, 17 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterForceMerge
   [junit4] Completed [235/454] on J2 in 2.14s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTieredMergePolicy
   [junit4] Completed [236/454] on J1 in 5.91s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestArrayUtil
   [junit4] Completed [237/454] on J2 in 3.51s, 14 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestBufferedIndexInput
   [junit4] Completed [238/454] on J2 in 0.40s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestConsistentFieldNumbers
   [junit4] Completed [239/454] on J2 in 2.04s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata
   [junit4] Completed [240/454] on J1 in 2.68s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDirectoryReader
   [junit4] Completed [241/454] on J1 in 3.66s, 24 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestLockFactory
   [junit4] Completed [242/454] on J1 in 0.18s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestSimpleFSLockFactory
   [junit4] Completed [243/454] on J0 in 11.20s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestBytesRef
   [junit4] Completed [244/454] on J0 in 0.06s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnVMError
   [junit4] IGNOR/A 0.00s J0 | TestIndexWriterOnVMError.testCheckpoint
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [245/454] on J0 in 1.03s, 3 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanNotQuery
   [junit4] Completed [246/454] on J0 in 0.06s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentMerger
   [junit4] Completed [247/454] on J1 in 2.14s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestWeakIdentityMap
   [junit4] Completed [248/454] on J2 in 9.35s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2
   [junit4] Completed [249/454] on J0 in 3.54s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFixedBitSet
   [junit4] Completed [250/454] on J2 in 3.32s, 19 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom2
   [junit4] Completed [251/454] on J2 in 2.59s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCollectionUtil
   [junit4] Completed [252/454] on J1 in 10.47s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentReader
   [junit4] Completed [253/454] on J1 in 1.34s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestNRTCachingDirectory
   [junit4] IGNOR/A 0.00s J2 | TestNRTCachingDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] IGNOR/A 0.00s J2 | TestNRTCachingDirectory.testPendingDeletions
   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory
   [junit4] Completed [254/454] on J2 in 3.02s, 44 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelCompositeReader
   [junit4] Completed [255/454] on J1 in 0.41s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.bkd.TestBKD
   [junit4] IGNOR/A 0.00s J0 | TestBKD.testRandomBinaryBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [256/454] on J0 in 13.29s, 19 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRewriteMethod
   [junit4] Completed [257/454] on J1 in 3.96s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestApproximationSearchEquivalence
   [junit4] Completed [258/454] on J0 in 1.32s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestLongValuesSource
   [junit4] Completed [259/454] on J0 in 2.17s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestPagedBytes
   [junit4] Completed [260/454] on J1 in 2.94s, 4 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTransactions
   [junit4] Completed [261/454] on J1 in 1.37s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCrash
   [junit4] Completed [262/454] on J1 in 0.63s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors
   [junit4] Completed [263/454] on J1 in 2.16s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentTermDocs
   [junit4] Completed [264/454] on J1 in 0.64s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterNRTIsCurrent
   [junit4] Completed [265/454] on J2 in 13.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDoubleRangeFieldQueries
   [junit4] IGNOR/A 0.00s J0 | TestDoubleRangeFieldQueries.testRandomBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [266/454] on J0 in 5.86s, 5 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiDocValues
   [junit4] Completed [267/454] on J2 in 0.23s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2
   [junit4] Completed [268/454] on J1 in 0.85s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFieldsReader
   [junit4] Completed [269/454] on J2 in 0.62s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocValuesIndexing
   [junit4] Completed [270/454] on J1 in 0.73s, 34 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarityBase
   [junit4] Completed [271/454] on J2 in 0.61s, 31 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
   [junit4] IGNOR/A 0.04s J0 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads
   [junit4]    > Assumption #1: aborting test: timeout obtaining lock
   [junit4] Completed [272/454] on J0 in 2.54s, 12 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiTermQueryRewrites
   [junit4] Completed [273/454] on J0 in 0.11s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanMinShouldMatch
   [junit4] Completed [274/454] on J1 in 4.00s, 17 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BDocs
   [junit4] IGNOR/A 0.00s J1 | Test2BDocs.test2BDocs
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~30min))
   [junit4] Completed [275/454] on J1 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminizeLexicon
   [junit4] Completed [276/454] on J2 in 4.14s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCustomNorms
   [junit4] Completed [277/454] on J1 in 0.86s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMixedCodecs
   [junit4] Completed [278/454] on J2 in 1.75s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFloatRangeFieldQueries
   [junit4] IGNOR/A 0.00s J0 | TestFloatRangeFieldQueries.testRandomBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [279/454] on J0 in 6.52s, 5 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressIndexing2
   [junit4] Completed [280/454] on J2 in 1.46s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestUTF32ToUTF8
   [junit4] Completed [281/454] on J1 in 4.34s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestRAMDirectory
   [junit4] IGNOR/A 0.01s J2 | TestRAMDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] IGNOR/A 0.00s J2 | TestRAMDirectory.testPendingDeletions
   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory
   [junit4] Completed [282/454] on J2 in 1.98s, 46 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestLazyProxSkipping
   [junit4] Completed [283/454] on J1 in 0.70s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRoaringDocIdSet
   [junit4] Completed [284/454] on J0 in 3.01s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMatchAllDocsQuery
   [junit4] Completed [285/454] on J0 in 0.07s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestWildcardRandom
   [junit4] Completed [286/454] on J0 in 0.22s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BTerms
   [junit4] IGNOR/A 0.00s J0 | Test2BTerms.test2BTerms
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=very slow, use 5g minimum heap))
   [junit4] Completed [287/454] on J0 in 0.02s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions
   [junit4] Completed [288/454] on J2 in 1.74s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestMergeSchedulerExternal
   [junit4] Completed [289/454] on J1 in 1.25s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermVectorsWriter
   [junit4] Completed [290/454] on J1 in 0.49s, 15 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCodecs
   [junit4] Completed [291/454] on J1 in 0.92s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitPositions
   [junit4] Completed [292/454] on J1 in 0.54s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBagOfPostings
   [junit4] Completed [293/454] on J2 in 4.51s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestBytesRefHash
   [junit4] Completed [294/454] on J1 in 2.75s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexInput
   [junit4] Completed [295/454] on J2 in 1.00s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestNearSpansOrdered
   [junit4] Completed [296/454] on J2 in 0.40s, 19 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitTf
   [junit4] Completed [297/454] on J1 in 0.52s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterDeleteQueue
   [junit4] Completed [298/454] on J2 in 0.10s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCrashCausesCorruptIndex
   [junit4] Completed [299/454] on J2 in 0.12s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocumentWriter
   [junit4] Completed [300/454] on J1 in 0.24s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiTermConstantScore
   [junit4] Completed [301/454] on J1 in 0.45s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanSearchEquivalence
   [junit4] Completed [302/454] on J1 in 2.79s, 33 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestScorerPerf
   [junit4] Completed [303/454] on J2 in 5.54s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPayloads
   [junit4] Completed [304/454] on J2 in 0.23s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarity2
   [junit4] Completed [305/454] on J2 in 0.33s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermVectorsReader
   [junit4] Completed [306/454] on J2 in 0.33s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIntBlockPool
   [junit4] Completed [307/454] on J2 in 0.69s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixInBooleanQuery
   [junit4] Completed [308/454] on J2 in 0.75s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentTermEnum
   [junit4] Completed [309/454] on J2 in 0.03s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanQuery
   [junit4] Completed [310/454] on J2 in 0.23s, 22 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
   [junit4] Completed [311/454] on J1 in 5.54s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestComplexExplanationsOfNonMatches
   [junit4] Completed [312/454] on J2 in 0.04s, 21 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestDocument
   [junit4] Completed [313/454] on J2 in 0.03s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTermScorer
   [junit4] Completed [314/454] on J2 in 0.04s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSumDocFreq
   [junit4] Completed [315/454] on J2 in 0.17s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexableField
   [junit4] Completed [316/454] on J2 in 0.49s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocValuesScoring
   [junit4] Completed [317/454] on J2 in 0.11s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiLevelSkipList
   [junit4] Completed [318/454] on J2 in 0.10s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestAutomatonQuery
   [junit4] Completed [319/454] on J1 in 1.95s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPayloadsOnVectors
   [junit4] Completed [320/454] on J2 in 0.07s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanQueryVisitSubscorers
   [junit4] Completed [321/454] on J1 in 0.24s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSizeBoundedForceMerge
   [junit4] Completed [322/454] on J2 in 0.17s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestConstantScoreQuery
   [junit4] Completed [323/454] on J2 in 0.10s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestCompiledAutomaton
   [junit4] Completed [324/454] on J1 in 0.26s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestElevationComparator
   [junit4] Completed [325/454] on J2 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBinaryTerms
   [junit4] Completed [326/454] on J1 in 0.18s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRecyclingIntBlockAllocator
   [junit4] Completed [327/454] on J2 in 0.06s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestConjunctions
   [junit4] Completed [328/454] on J1 in 0.17s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimilarity
   [junit4] Completed [329/454] on J2 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanationsOfNonMatches
   [junit4] Completed [330/454] on J1 in 0.10s, 30 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSmallFloat
   [junit4] Completed [331/454] on J1 in 0.16s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRollingBuffer
   [junit4] Completed [332/454] on J2 in 0.34s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimilarityProvider
   [junit4] Completed [333/454] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFilterIterator
   [junit4] Completed [334/454] on J2 in 0.02s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestDateTools
   [junit4] Completed [335/454] on J1 in 0.02s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCharsRef
   [junit4] Completed [336/454] on J2 in 0.02s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BPostings
   [junit4] IGNOR/A 0.00s J2 | Test2BPostings.test
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [337/454] on J2 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestPriorityQueue
   [junit4] Completed [338/454] on J1 in 0.46s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestCachingCollector
   [junit4] Completed [339/454] on J2 in 0.03s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestBinaryDocument
   [junit4] Completed [340/454] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTotalHitCountCollector
   [junit4] Completed [341/454] on J2 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimator
   [junit4] IGNOR/A 0.00s J1 | TestRamUsageEstimator.testPrintValues
   [junit4]    > Assumption #1: Specify -Dtests.verbose=true to print constants of RamUsageEstimator.
   [junit4] Completed [342/454] on J1 in 0.01s, 5 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanFirstQuery
   [junit4] Completed [343/454] on J2 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIsCurrent
   [junit4] Completed [344/454] on J1 in 0.02s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNoMergeScheduler
   [junit4] Completed [345/454] on J2 in 0.01s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterLockRelease
   [junit4] Completed [346/454] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestByteBlockPool
   [junit4] Completed [347/454] on J2 in 0.06s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestDemo
   [junit4] Completed [348/454] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestRollback
   [junit4] Completed [349/454] on J2 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCloseableThreadLocal
   [junit4] Completed [350/454] on J1 in 0.01s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCodecHoldsOpenFiles
   [junit4] Completed [351/454] on J2 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestSimpleAttributeImpl
   [junit4] Completed [352/454] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexCommit
   [junit4] Completed [353/454] on J2 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTerm
   [junit4] Completed [354/454] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BNumericDocValues
   [junit4] IGNOR/A 0.00s J2 | Test2BNumericDocValues.testNumerics
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 2 hours if the heap is 5gb))
   [junit4] Completed [355/454] on J2 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCharArrayMap
   [junit4] Completed [356/454] on J1 in 0.10s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCharArraySet
   [junit4] Completed [357/454] on J2 in 0.03s, 15 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestDelegatingAnalyzerWrapper
   [junit4] Completed [358/454] on J1 in 0.02s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestReusableStringReader
   [junit4] Completed [359/454] on J2 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestStopFilter
   [junit4] Completed [360/454] on J1 in 0.01s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestWordlistLoader
   [junit4] Completed [361/454] on J2 in 0.01s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.TestCodecLoadingDeadlock
   [junit4] Completed [362/454] on J1 in 0.60s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormat
   [junit4] IGNOR/A 0.00s J1 | TestLucene50StoredFieldsFormat.testBigDocuments
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [363/454] on J1 in 3.05s, 17 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat
   [junit4] Completed [364/454] on J2 in 5.74s, 27 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene70.TestIndexedDISI
   [junit4] Completed [365/454] on J1 in 6.01s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermsEnum
   [junit4] Completed [366/454] on J0 in 44.42s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFlex
   [junit4] Completed [367/454] on J0 in 0.26s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMaxTermFrequency
   [junit4] Completed [368/454] on J0 in 0.45s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene70.TestLucene70NormsFormat
   [junit4] IGNOR/A 0.00s J1 | TestLucene70NormsFormat.testMergeStability
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] IGNOR/A 0.00s J1 | TestLucene70NormsFormat.testNCommonBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J1 | TestLucene70NormsFormat.testSparseNCommonBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [369/454] on J1 in 16.35s, 29 tests, 3 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene70.TestLucene70SegmentInfoFormat
   [junit4] Completed [370/454] on J1 in 0.54s, 16 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestFieldType
   [junit4] Completed [371/454] on J1 in 0.03s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50CompoundFormat
   [junit4] Completed [372/454] on J0 in 7.16s, 26 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.geo.TestGeoEncodingUtils
   [junit4] Completed [373/454] on J1 in 1.52s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.geo.TestPolygon
   [junit4] Completed [374/454] on J1 in 0.11s, 15 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BPoints
   [junit4] IGNOR/A 0.00s J1 | Test2BPoints.test1D
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))
   [junit4] IGNOR/A 0.00s J1 | Test2BPoints.test2D
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))
   [junit4] Completed [375/454] on J1 in 0.02s, 2 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesFixedSorted
   [junit4] IGNOR/A 0.00s J1 | Test2BSortedDocValuesFixedSorted.testFixedSorted
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))
   [junit4] Completed [376/454] on J1 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesOrds
   [junit4] IGNOR/A 0.00s J1 | Test2BSortedDocValuesOrds.test2BOrds
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))
   [junit4] Completed [377/454] on J1 in 0.01s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesDetectTruncation
   [junit4] Completed [378/454] on J1 in 0.60s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveChecksumFooter
   [junit4] Completed [379/454] on J1 in 0.44s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCustomTermFreq
   [junit4] Completed [380/454] on J1 in 0.23s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDemoParallelLeafReader
   [junit4] Completed [381/454] on J1 in 8.91s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocIDMerger
   [junit4] Completed [382/454] on J1 in 0.05s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocValues
   [junit4] Completed [383/454] on J1 in 0.15s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestExceedMaxTermLength
   [junit4] Completed [384/454] on J1 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFieldInvertState
   [junit4] Completed [385/454] on J1 in 0.14s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.geo.TestGeoUtils
   [junit4] Completed [386/454] on J0 in 14.08s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFilterCodecReader
   [junit4] Completed [387/454] on J1 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexReaderClose
   [junit4] Completed [388/454] on J0 in 0.40s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions2
   [junit4] Completed [389/454] on J0 in 5.69s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterFromReader
   [junit4] Completed [390/454] on J0 in 0.28s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestManyFields
   [junit4] Completed [391/454] on J0 in 2.15s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMaxPosition
   [junit4] Completed [392/454] on J0 in 0.02s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMergeRateLimiter
   [junit4] Completed [393/454] on J0 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiTermsEnum
   [junit4] Completed [394/454] on J0 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNRTReaderCleanup
   [junit4] Completed [395/454] on J0 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexSorting
   [junit4] Completed [396/454] on J1 in 18.33s, 50 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOneMergeWrappingMergePolicy
   [junit4] Completed [397/454] on J1 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOrdinalMap
   [junit4] Completed [398/454] on J1 in 0.16s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNumericDocValuesUpdates
   [junit4] Completed [399/454] on J0 in 19.34s, 32 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSortedSetDocValues
   [junit4] Completed [400/454] on J0 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSwappedIndexFiles
   [junit4] Completed [401/454] on J0 in 0.09s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTragicIndexWriterDeadlock
   [junit4] Completed [402/454] on J0 in 0.30s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestUpgradeIndexMergePolicy
   [junit4] Completed [403/454] on J0 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.FuzzyTermOnShortTermsTest
   [junit4] Completed [404/454] on J0 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TermInSetQueryTest
   [junit4] Completed [405/454] on J0 in 0.75s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPointValues
   [junit4] Completed [406/454] on J1 in 13.24s, 32 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBlendedTermQuery
   [junit4] Completed [407/454] on J0 in 0.04s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBoolean2ScorerSupplier
   [junit4] Completed [408/454] on J1 in 0.23s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBoostQuery
   [junit4] Completed [409/454] on J0 in 0.04s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestControlledRealTimeReopenThread
   [junit4] IGNOR/A 0.00s J1 | TestControlledRealTimeReopenThread.testCRTReopen
   [junit4]    > Assumption #1: 'awaitsfix' test group is disabled (@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/LUCENE-5737))
   [junit4] Completed [410/454] on J1 in 3.91s, 6 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestEarlyTermination
   [junit4] Completed [411/454] on J1 in 0.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFieldValueQuery
   [junit4] Completed [412/454] on J1 in 2.88s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestIndexOrDocValuesQuery
   [junit4] Completed [413/454] on J1 in 0.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestIntRangeFieldQueries
   [junit4] IGNOR/A 0.00s J1 | TestIntRangeFieldQueries.testRandomBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [414/454] on J1 in 6.52s, 5 tests, 1 s

[...truncated too long message...]

jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

common.resolve:

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null645443369
     [copy] Copying 35 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null645443369
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/null645443369

resolve-example:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null361998183
     [copy] Copying 214 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null361998183
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/null361998183

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 30ms :: artifacts dl 2ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 92 minutes 8 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-Tests-7.x - Build # 111 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/111/

2 tests failed.
FAILED:  org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread.testPublishDownState

Error Message:
Could not load collection from ZK: collection1

Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
	at __randomizedtesting.SeedInfo.seed([8615D0B12D471F74:D868724F895A5689]:0)
	at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
	at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
	at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:213)
	at org.apache.solr.common.cloud.ClusterState.getSlicesMap(ClusterState.java:153)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:901)
	at org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread.testPublishDownState(TestLeaderInitiatedRecoveryThread.java:125)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
	at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1125)
	at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1111)
	... 49 more


FAILED:  org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([8615D0B12D471F74:AA27A73CA18369F0]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:264)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=int_i:21&wt=xml
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
	... 40 more




Build Log:
[...truncated 11054 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestLeaderInitiatedRecoveryThread
   [junit4]   2> 409750 INFO  (SUITE-TestLeaderInitiatedRecoveryThread-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/init-core-data-001
   [junit4]   2> 409751 INFO  (SUITE-TestLeaderInitiatedRecoveryThread-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 409752 INFO  (SUITE-TestLeaderInitiatedRecoveryThread-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 409752 INFO  (SUITE-TestLeaderInitiatedRecoveryThread-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /wzuzp/
   [junit4]   2> 409863 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 409865 INFO  (Thread-248) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 409865 INFO  (Thread-248) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 409868 ERROR (Thread-248) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 409965 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ZkTestServer start zk server on port:34348
   [junit4]   2> 409974 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 409975 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 409977 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 409978 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 409978 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 409979 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 409980 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 409993 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 409994 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 409995 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 409996 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 409998 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 410093 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 410094 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b8d16c4{/wzuzp,null,AVAILABLE}
   [junit4]   2> 410101 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@21767d16{HTTP/1.1,[http/1.1]}{127.0.0.1:39767}
   [junit4]   2> 410101 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.Server Started @416863ms
   [junit4]   2> 410101 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/tempDir-001/control/data, hostContext=/wzuzp, hostPort=39767, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/control-001/cores}
   [junit4]   2> 410102 ERROR (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 410102 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 410102 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 410102 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 410102 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-29T00:33:29.758Z
   [junit4]   2> 410125 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 410125 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/control-001/solr.xml
   [junit4]   2> 410141 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 410151 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34348/solr
   [junit4]   2> 410276 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 410276 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39767_wzuzp
   [junit4]   2> 410277 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.Overseer Overseer (id=98388437206433796-127.0.0.1:39767_wzuzp-n_0000000000) starting
   [junit4]   2> 410282 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39767_wzuzp
   [junit4]   2> 410330 INFO  (OverseerStateUpdate-98388437206433796-127.0.0.1:39767_wzuzp-n_0000000000) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 414637 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:39767_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 414657 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:39767_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 414657 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:39767_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 414658 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/control-001/cores
   [junit4]   2> 414722 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 414722 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34348/solr ready
   [junit4]   2> 414725 INFO  (qtp844015017-1154) [n:127.0.0.1:39767_wzuzp    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39767_wzuzp&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 414738 INFO  (OverseerThreadFactory-616-thread-1-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 414880 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 414880 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 415006 INFO  (zkCallback-187-thread-2-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 415981 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 416028 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 416585 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 416733 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 416747 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 416748 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 416748 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 416750 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=5.6220703125, floorSegmentMB=0.998046875, forceMergeDeletesPctAllowed=5.811941858157466, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4558896500132007
   [junit4]   2> 416757 WARN  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 416947 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 416947 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 416948 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 416948 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 416949 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=35.80078125, floorSegmentMB=0.4853515625, forceMergeDeletesPctAllowed=6.862892492883461, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5101820297413789
   [junit4]   2> 416950 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@73424dcf[control_collection_shard1_replica_n1] main]
   [junit4]   2> 416975 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 416975 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 416976 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 416976 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574215002758316032
   [junit4]   2> 416989 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 416989 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 416989 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39767/wzuzp/control_collection_shard1_replica_n1/
   [junit4]   2> 416989 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 416989 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39767/wzuzp/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 416989 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 417061 INFO  (searcherExecutor-619-thread-1-processing-n:127.0.0.1:39767_wzuzp x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@73424dcf[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 417110 INFO  (zkCallback-187-thread-2-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 417125 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39767/wzuzp/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 417227 INFO  (zkCallback-187-thread-2-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 417295 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 417296 INFO  (qtp844015017-1152) [n:127.0.0.1:39767_wzuzp c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2416
   [junit4]   2> 417458 INFO  (zkCallback-187-thread-2-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 417479 INFO  (qtp844015017-1154) [n:127.0.0.1:39767_wzuzp    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 417479 INFO  (qtp844015017-1154) [n:127.0.0.1:39767_wzuzp    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39767_wzuzp&wt=javabin&version=2} status=0 QTime=2754
   [junit4]   2> 417480 INFO  (OverseerCollectionConfigSetProcessor-98388437206433796-127.0.0.1:39767_wzuzp-n_0000000000) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 417775 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 417775 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34348/solr ready
   [junit4]   2> 417775 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 417777 INFO  (qtp844015017-1155) [n:127.0.0.1:39767_wzuzp    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 417795 INFO  (OverseerThreadFactory-616-thread-2-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 417796 WARN  (OverseerThreadFactory-616-thread-2-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 418268 INFO  (qtp844015017-1155) [n:127.0.0.1:39767_wzuzp    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 418268 INFO  (qtp844015017-1155) [n:127.0.0.1:39767_wzuzp    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=491
   [junit4]   2> 419817 INFO  (OverseerCollectionConfigSetProcessor-98388437206433796-127.0.0.1:39767_wzuzp-n_0000000000) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 420103 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-1-001 of type NRT
   [junit4]   2> 420104 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 420138 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@24654690{/wzuzp,null,AVAILABLE}
   [junit4]   2> 420139 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@436cae09{HTTP/1.1,[http/1.1]}{127.0.0.1:41045}
   [junit4]   2> 420139 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.Server Started @426901ms
   [junit4]   2> 420139 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/wzuzp, hostPort=41045, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-1-001/cores}
   [junit4]   2> 420139 ERROR (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 420139 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 420139 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 420139 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 420139 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-29T00:33:39.795Z
   [junit4]   2> 420609 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 420609 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-1-001/solr.xml
   [junit4]   2> 420662 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 420679 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34348/solr
   [junit4]   2> 420926 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 420927 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 420928 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41045_wzuzp
   [junit4]   2> 421047 INFO  (zkCallback-187-thread-1-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 421055 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 421149 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 421422 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:41045_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 421426 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:41045_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 421427 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:41045_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 421428 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-1-001/cores
   [junit4]   2> 421485 INFO  (qtp844015017-1158) [n:127.0.0.1:39767_wzuzp    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:41045_wzuzp&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 421489 INFO  (OverseerThreadFactory-616-thread-3-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:41045_wzuzp for creating new replica
   [junit4]   2> 421500 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 421500 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 421504 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 422512 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 422547 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 422677 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 422719 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
   [junit4]   2> 422719 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 422719 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 422719 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-1-001/cores/collection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-1-001/cores/collection1_shard1_replica_n1/data/]
   [junit4]   2> 422722 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=5.6220703125, floorSegmentMB=0.998046875, forceMergeDeletesPctAllowed=5.811941858157466, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4558896500132007
   [junit4]   2> 422745 WARN  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 422862 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 422863 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 422863 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 422863 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 422864 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=35.80078125, floorSegmentMB=0.4853515625, forceMergeDeletesPctAllowed=6.862892492883461, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5101820297413789
   [junit4]   2> 422865 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d21257c[collection1_shard1_replica_n1] main]
   [junit4]   2> 422866 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 422868 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 422868 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 422868 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574215008936525824
   [junit4]   2> 422870 INFO  (searcherExecutor-630-thread-1-processing-n:127.0.0.1:41045_wzuzp x:collection1_shard1_replica_n1 s:shard1 c:collection1) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@2d21257c[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 422884 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 422884 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 422884 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41045/wzuzp/collection1_shard1_replica_n1/
   [junit4]   2> 422884 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 422884 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:41045/wzuzp/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 422884 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 422887 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41045/wzuzp/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 422989 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 423038 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 423039 INFO  (qtp1475136319-1197) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1538
   [junit4]   2> 423041 INFO  (qtp844015017-1158) [n:127.0.0.1:39767_wzuzp    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41045_wzuzp&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1555
   [junit4]   2> 423356 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 423489 INFO  (OverseerCollectionConfigSetProcessor-98388437206433796-127.0.0.1:39767_wzuzp-n_0000000000) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 423790 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-2-001 of type NRT
   [junit4]   2> 423791 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 423804 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@311cb9df{/wzuzp,null,AVAILABLE}
   [junit4]   2> 423804 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@925b721{HTTP/1.1,[http/1.1]}{127.0.0.1:46249}
   [junit4]   2> 423818 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.Server Started @430580ms
   [junit4]   2> 423818 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/wzuzp, hostPort=46249, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-2-001/cores}
   [junit4]   2> 423818 ERROR (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 423819 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 423819 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 423819 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 423819 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-29T00:33:43.475Z
   [junit4]   2> 423840 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 423841 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-2-001/solr.xml
   [junit4]   2> 423846 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 423888 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34348/solr
   [junit4]   2> 423993 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 424010 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 424029 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46249_wzuzp
   [junit4]   2> 424030 INFO  (zkCallback-187-thread-1-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 424031 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 424083 INFO  (zkCallback-205-thread-1-processing-n:127.0.0.1:46249_wzuzp) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 424083 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 424364 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 424384 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 424385 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 424386 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-2-001/cores
   [junit4]   2> 424520 INFO  (qtp844015017-1151) [n:127.0.0.1:39767_wzuzp    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46249_wzuzp&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 424562 INFO  (OverseerThreadFactory-616-thread-4-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:46249_wzuzp for creating new replica
   [junit4]   2> 424580 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 424581 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 424699 INFO  (zkCallback-205-thread-1-processing-n:127.0.0.1:46249_wzuzp) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 424700 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 425676 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 425751 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.s.IndexSchema [collection1_shard1_replica_n3] Schema name=test
   [junit4]   2> 430919 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 431009 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n3' using configuration from collection collection1, trusted=true
   [junit4]   2> 431022 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n3' (registry 'solr.core.collection1.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 431022 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 431022 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.SolrCore [[collection1_shard1_replica_n3] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-2-001/cores/collection1_shard1_replica_n3], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-2-001/cores/collection1_shard1_replica_n3/data/]
   [junit4]   2> 431024 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=5.6220703125, floorSegmentMB=0.998046875, forceMergeDeletesPctAllowed=5.811941858157466, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4558896500132007
   [junit4]   2> 431044 WARN  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 431448 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 431448 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 431449 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 431449 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 431451 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=35.80078125, floorSegmentMB=0.4853515625, forceMergeDeletesPctAllowed=6.862892492883461, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5101820297413789
   [junit4]   2> 431452 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@752e23e3[collection1_shard1_replica_n3] main]
   [junit4]   2> 431521 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 431537 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 431538 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 431538 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574215018027679744
   [junit4]   2> 431559 INFO  (searcherExecutor-641-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.SolrCore [collection1_shard1_replica_n3] Registered new searcher Searcher@752e23e3[collection1_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 431667 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n3
   [junit4]   2> 431704 INFO  (qtp555991869-1225) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=7123
   [junit4]   2> 431716 INFO  (updateExecutor-202-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 431716 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 431732 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 431732 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n3]
   [junit4]   2> 431732 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 431732 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_n3] as recovering, leader is [http://127.0.0.1:41045/wzuzp/collection1_shard1_replica_n1/] and I am [http://127.0.0.1:46249/wzuzp/collection1_shard1_replica_n3/]
   [junit4]   2> 431794 INFO  (qtp844015017-1151) [n:127.0.0.1:39767_wzuzp    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:46249_wzuzp&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=7273
   [junit4]   2> 431809 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:41045/wzuzp]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_n1&nodeName=127.0.0.1:46249_wzuzp&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 431810 INFO  (zkCallback-205-thread-1-processing-n:127.0.0.1:46249_wzuzp) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 431829 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 431846 INFO  (qtp1475136319-1195) [n:127.0.0.1:41045_wzuzp    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 431870 INFO  (qtp1475136319-1195) [n:127.0.0.1:41045_wzuzp    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:46249_wzuzp, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1_shard1_replica_n3","base_url":"http://127.0.0.1:46249/wzuzp","node_name":"127.0.0.1:46249_wzuzp","state":"recovering","type":"NRT"}
   [junit4]   2> 431870 INFO  (qtp1475136319-1195) [n:127.0.0.1:41045_wzuzp    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
   [junit4]   2> 431870 INFO  (qtp1475136319-1195) [n:127.0.0.1:41045_wzuzp    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:46249_wzuzp&onlyIfLeaderActive=true&core=collection1_shard1_replica_n1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=24
   [junit4]   2> 431871 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testPublishDownState
   [junit4]   2> 431871 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
   [junit4]   1> -
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:recovering live:true
   [junit4]   2> 432414 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:41045/wzuzp/collection1_shard1_replica_n1/] - recoveringAfterStartup=[true]
   [junit4]   2> 432414 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n3 url=http://127.0.0.1:46249/wzuzp START replicas=[http://127.0.0.1:41045/wzuzp/collection1_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 432452 INFO  (qtp1475136319-1200) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 432452 INFO  (qtp1475136319-1200) [n:127.0.0.1:41045_wzuzp c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp=/wzuzp path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 432453 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 432453 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync 
   [junit4]   2> 432453 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 432453 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 432454 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 432454 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
   [junit4]   2> 432454 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
   [junit4]   2> 432454 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy No replay needed.
   [junit4]   2> 432454 INFO  (recoveryExecutor-203-thread-1-processing-n:127.0.0.1:46249_wzuzp x:collection1_shard1_replica_n3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 432565 INFO  (zkCallback-205-thread-1-processing-n:127.0.0.1:46249_wzuzp) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 432580 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 432728 INFO  (OverseerCollectionConfigSetProcessor-98388437206433796-127.0.0.1:39767_wzuzp-n_0000000000) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   1> -
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   2> 432873 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 432894 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ChaosMonkey monkey: stop jetty! 46249
   [junit4]   2> 432894 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1819104834
   [junit4]   2> 432894 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 432895 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 432895 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 432897 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 432942 INFO  (coreCloseExecutor-646-thread-1) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.SolrCore [collection1_shard1_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@196d5088
   [junit4]   2> 432970 INFO  (coreCloseExecutor-646-thread-1) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n3, tag=426594440
   [junit4]   2> 432973 INFO  (coreCloseExecutor-646-thread-1) [n:127.0.0.1:46249_wzuzp c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=426594440
   [junit4]   2> 433001 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 433012 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 433012 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 433019 INFO  (zkCallback-187-thread-2-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 433032 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@925b721{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 433032 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@311cb9df{/wzuzp,null,UNAVAILABLE}
   [junit4]   2> 433069 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.LeaderInitiatedRecoveryThread Node 127.0.0.1:46249_wzuzp is not live, so skipping leader-initiated recovery for replica: core=collection1_shard1_replica_n3 coreNodeName=core_node4
   [junit4]   2> 433069 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.LeaderInitiatedRecoveryThread Node 127.0.0.1:46249_wzuzp is not live, so skipping leader-initiated recovery for replica: core=collection1_shard1_replica_n3 coreNodeName=core_node4
   [junit4]   2> 433069 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ChaosMonkey monkey: starting jetty! 46249
   [junit4]   2> 433070 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 433231 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@61c4cd3{/wzuzp,null,AVAILABLE}
   [junit4]   2> 433231 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@66f7cdd7{HTTP/1.1,[http/1.1]}{127.0.0.1:46249}
   [junit4]   2> 433231 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.e.j.s.Server Started @439993ms
   [junit4]   2> 433232 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/wzuzp, hostPort=46249, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-2-001/cores}
   [junit4]   2> 433232 ERROR (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 433232 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 433233 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 433233 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 433233 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-29T00:33:52.889Z
   [junit4]   2> 433289 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 433289 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderInitiatedRecoveryThread_8615D0B12D471F74-001/shard-2-001/solr.xml
   [junit4]   2> 433321 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 433373 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34348/solr
   [junit4]   2> 433490 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 433492 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 433493 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46249_wzuzp
   [junit4]   2> 433575 INFO  (zkCallback-212-thread-1-processing-n:127.0.0.1:46249_wzuzp) [n:127.0.0.1:46249_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 433624 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 433624 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 433624 INFO  (zkCallback-187-thread-2-processing-n:127.0.0.1:39767_wzuzp) [n:127.0.0.1:39767_wzuzp    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 433944 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:41045_wzuzp) [n:127.0.0.1:41045_wzuzp    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 434415 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 434453 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34597c0d
   [junit4]   2> 434453 INFO  (TEST-TestLeaderInitiatedRecoveryThread.testPublishDownState-seed#[8615D0B12D471F74]) [n:127.0.0.1:46249_wzuzp    ] o.a.s.m.r.SolrJmxReporter JMX moni

[...truncated too long message...]

 path=null params={q=cat:human&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 3934643 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[8615D0B12D471F74]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:animal&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 3934643 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[8615D0B12D471F74]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=title:Mr&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 3934644 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[8615D0B12D471F74]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=title:Dr&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 3934644 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[8615D0B12D471F74]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=count_i:20&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 3934652 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[8615D0B12D471F74]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=count_i:40&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 3934653 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[8615D0B12D471F74]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name_s:Virat&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 3934653 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[8615D0B12D471F74]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=multiDefault:Delhi&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 3934699 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[8615D0B12D471F74]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBasics
   [junit4]   2> 3934700 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 3934700 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=878813912
   [junit4]   2> 3934700 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 3934701 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 3934702 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 3934717 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[8615D0B12D471F74]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3934730 INFO  (coreCloseExecutor-10085-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1f126d6c
   [junit4]   2> 3934778 INFO  (coreCloseExecutor-10085-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=521301356
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.update.processor.AtomicUpdateProcessorFactoryTest_8615D0B12D471F74-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=sq-AL, timezone=PST8PDT
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=303163432,total=530055168
   [junit4]   2> NOTE: All tests run in this JVM: [BadIndexSchemaTest, PreAnalyzedFieldTest, CoreAdminRequestStatusTest, CloudMLTQParserTest, TestReload, TestRebalanceLeaders, StatsComponentTest, LargeFieldTest, TestSolrDeletionPolicy2, HdfsTlogReplayBufferedWhileIndexingTest, DocValuesNotIndexedTest, URLClassifyProcessorTest, ClusterStateTest, TestFieldResource, PingRequestHandlerTest, SpatialHeatmapFacetsTest, TriLevelCompositeIdRoutingTest, SpellPossibilityIteratorTest, TestFoldingMultitermQuery, TestChildDocTransformer, TestSmileRequest, UnloadDistributedZkTest, TestAnalyzedSuggestions, TestLocalFSCloudBackupRestore, TestHdfsBackupRestoreCore, TestJmxIntegration, TestDistributedMissingSort, DirectoryFactoryTest, TestCloudDeleteByQuery, TestSha256AuthenticationProvider, CollectionReloadTest, TestJsonFacetRefinement, RequiredFieldsTest, TestZkChroot, TestSubQueryTransformerDistrib, OutOfBoxZkACLAndCredentialsProvidersTest, RulesTest, PolyFieldTest, TestTestInjection, SuggesterFSTTest, SegmentsInfoRequestHandlerTest, GraphQueryTest, TestUtils, JvmMetricsTest, TestStreamBody, ReturnFieldsTest, BufferStoreTest, FastVectorHighlighterTest, TestDistributedStatsComponentCardinality, NumericFieldsTest, TestPseudoReturnFields, RecoveryZkTest, TestCustomDocTransformer, TestStressInPlaceUpdates, DistributedQueryComponentOptimizationTest, SuggesterTest, TestGroupingSearch, TestOrdValues, BadCopyFieldTest, TestGraphMLResponseWriter, TestDFISimilarityFactory, TestSuggestSpellingConverter, TestBlendedInfixSuggestions, SolrSlf4jReporterTest, HLLUtilTest, ResponseLogComponentTest, TestSolrFieldCacheBean, TestCloudManagedSchema, TestPayloadScoreQParserPlugin, CopyFieldTest, TestTrie, TestObjectReleaseTracker, MinimalSchemaTest, AnalyticsMergeStrategyTest, TestSolrConfigHandlerCloud, TestComplexPhraseLeadingWildcard, TestManagedSchemaThreadSafety, ClassificationUpdateProcessorFactoryTest, DistributedFacetPivotLargeTest, MissingSegmentRecoveryTest, CircularListTest, SaslZkACLProviderTest, AliasIntegrationTest, TestMaxScoreQueryParser, ConfigSetsAPITest, PeerSyncWithIndexFingerprintCachingTest, PrimitiveFieldTypeTest, TestQueryUtils, TestLeaderElectionWithEmptyReplica, OverseerModifyCollectionTest, TestManagedStopFilterFactory, EnumFieldTest, SortByFunctionTest, MoveReplicaTest, BasicDistributedZk2Test, JSONWriterTest, TestDocTermOrds, TestExactSharedStatsCache, ZkStateWriterTest, TestJettySolrRunner, CachingDirectoryFactoryTest, TimeZoneUtilsTest, TestConfigSetsAPI, TestSchemaNameResource, LeaderElectionTest, TestDownShardTolerantSearch, TestDFRSimilarityFactory, TestCodecSupport, ParsingFieldUpdateProcessorsTest, TestSolrCloudWithHadoopAuthPlugin, CdcrReplicationDistributedZkTest, TestCloudPivotFacet, DirectUpdateHandlerTest, ConjunctionSolrSpellCheckerTest, OpenCloseCoreStressTest, OverseerTest, BasicZkTest, TestRandomFaceting, ZkCLITest, AutoCommitTest, TestRecovery, ZkControllerTest, TestRealTimeGet, TestStressReorder, HardAutoCommitTest, SimpleFacetsTest, TestSolr4Spatial, QueryElevationComponentTest, ConvertedLegacyTest, TestSort, TestFiltering, TestIndexSearcher, HighlighterTest, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, CoreAdminHandlerTest, SuggesterTSTTest, SpatialFilterTest, SuggesterWFSTTest, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, DocValuesMultiTest, SolrInfoBeanTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestReversedWildcardFilterFactory, TestQueryTypes, TermsComponentTest, DocumentBuilderTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TestSolrQueryParser, TestCollationField, BinaryUpdateRequestHandlerTest, TestComponentsName, TestLFUCache, CursorPagingTest, AsyncCallRequestStatusResponseTest, CdcrBootstrapTest, CleanupOldIndexTest, CollectionsAPIAsyncDistributedZkTest, ConcurrentDeleteAndCreateCollectionTest, CreateCollectionCleanupTest, CustomCollectionTest, DeleteReplicaTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, ForceLeaderTest, LeaderInitiatedRecoveryOnCommitTest, MoveReplicaHDFSTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, RecoveryAfterSoftCommitTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCloudRecovery, TestMiniSolrCloudClusterSSL, TestPullReplicaErrorHandling, TestRandomFlRTGCloud, TestReplicaProperties, TestShortCircuitedRequests, TestSolrCloudWithDelegationTokens, TestSolrCloudWithKerberosAlt, TestStressLiveNodes, TestTlogReplica, TestImplicitCoreProperties, TestInfoStreamLogging, TestInitParams, TestNRTOpen, TestShardHandlerFactory, TestSimpleTextCodec, TestSolrConfigHandler, TestSolrCloudSnapshots, CoreMergeIndexesAdminHandlerTest, MetricsHandlerTest, PropertiesRequestHandlerTest, StatsReloadRaceTest, DistributedFacetExistsSmallTest, DistributedFacetPivotSmallTest, ReplicaListTransformerTest, ShufflingReplicaListTransformerTest, SuggestComponentTest, TestNumericRangeQuery64, SolrMetricManagerTest, SolrGangliaReporterTest, SolrShardReporterTest, TestJavabinTupleStreamParser, TestPushWriter, TestRawResponseWriter, TestRawTransformer, TestSolrQueryResponse, TestRestManager, TestDynamicFieldCollectionResource, TestSchemaResource, TestSerializedLuceneMatchVersion, BooleanFieldTest, CurrencyFieldTypeTest, TestCloudSchemaless, TestMinMaxOnMultiValuedField, BlockJoinFacetDistribTest, BlockJoinFacetSimpleTest, TestScoreJoinQPNoScore, TestDefaultStatsCache, TestExactStatsCache, BasicAuthIntegrationTest, HttpSolrCallGetCoreTest, BlockDirectoryTest, TestFieldCacheReopen, TestFieldCacheSort, TestFieldCacheSortRandom, TestFieldCacheWithThreads, TestNumericTerms32, TestUninvertingReader, AnalysisErrorHandlingTest, CdcrUpdateLogTest, TestExceedMaxTermLength, TestHdfsUpdateLog, AddSchemaFieldsUpdateProcessorFactoryTest, AtomicUpdateProcessorFactoryTest]
   [junit4] Completed [705/726 (2!)] on J2 in 1.34s, 4 tests, 1 error <<< FAILURES!

[...truncated 43340 lines...]

[JENKINS] Lucene-Solr-Tests-7.x - Build # 110 - Still Unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/110/

1 tests failed.
FAILED:  org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([AA77EA985CEFFD2E:86459D15D02B8BAA]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:264)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=int_i:11&wt=xml
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
	... 40 more




Build Log:
[...truncated 11263 lines...]
   [junit4] Suite: org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest
   [junit4]   2> 660902 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.update.processor.AtomicUpdateProcessorFactoryTest_AA77EA985CEFFD2E-001/init-core-data-001
   [junit4]   2> 660910 WARN  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=717 numCloses=717
   [junit4]   2> 660910 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 660911 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 660912 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 660912 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 660994 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 661073 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 661415 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 661514 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b06c192
   [junit4]   2> 661529 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b06c192
   [junit4]   2> 661529 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b06c192
   [junit4]   2> 661531 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 661616 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 661694 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 661963 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 661969 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 661969 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b06c192
   [junit4]   2> 661969 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 661969 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.update.processor.AtomicUpdateProcessorFactoryTest_AA77EA985CEFFD2E-001/init-core-data-001/]
   [junit4]   2> 661987 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.654296875, noCFSRatio=0.0]
   [junit4]   2> 662106 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 662106 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 662107 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 662107 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 662108 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=19, maxMergedSegmentMB=2.5009765625, floorSegmentMB=0.3701171875, forceMergeDeletesPctAllowed=10.554544450733975, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 662108 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4149f3e7[collection1] main]
   [junit4]   2> 662109 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 662139 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 662202 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 662202 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 662202 INFO  (coreLoadExecutor-1246-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574170269259399168
   [junit4]   2> 662203 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 662203 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 662203 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 662203 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 662203 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 662203 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 662203 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 662203 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 662205 INFO  (searcherExecutor-1247-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4149f3e7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 662206 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 662218 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasics
   [junit4]   2> 662222 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1574170269280370688,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 662222 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5c3f69a8 commitCommandVersion:1574170269280370688
   [junit4]   2> 662226 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1c04d19d[collection1] main]
   [junit4]   2> 662226 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 662227 INFO  (searcherExecutor-1247-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1c04d19d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C1)))}
   [junit4]   2> 662227 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 5
   [junit4]   2> 662230 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:1&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662230 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:human&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662231 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=title:Mr&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662231 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=count_i:20&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662232 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name_s:Virat&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662232 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=multiDefault:Delhi&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662250 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1574170269309730816,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 662250 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5c3f69a8 commitCommandVersion:1574170269309730816
   [junit4]   2> 662253 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@44394e1f[collection1] main]
   [junit4]   2> 662253 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 662257 INFO  (searcherExecutor-1247-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@44394e1f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(7.1.0):C1)))}
   [junit4]   2> 662258 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 24
   [junit4]   2> 662266 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:1&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662267 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:human&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662267 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:animal&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662268 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=title:Mr&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 662268 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=title:Dr&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662268 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=count_i:20&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 662269 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=count_i:40&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662269 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name_s:Virat&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 662269 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=multiDefault:Delhi&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 662270 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBasics
   [junit4]   2> 662295 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testNoUniqueIdPassed-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNoUniqueIdPassed
   [junit4]   2> 662296 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testNoUniqueIdPassed-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNoUniqueIdPassed
   [junit4]   2> 662297 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testWrongAtomicOpPassed-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testWrongAtomicOpPassed
   [junit4]   2> 662297 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testWrongAtomicOpPassed-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testWrongAtomicOpPassed
   [junit4]   2> 662298 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMultipleThreads
   [junit4]   2> 662299 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 662299 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@5436174d[collection1] realtime]
   [junit4]   2> 662299 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
   [junit4]   2> 662340 WARN  (Thread-666) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269362159616 actual=1574170269403054080Retrying with new version .... (1)
   [junit4]   2> 662340 WARN  (Thread-665) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269362159616 actual=1574170269403054080Retrying with new version .... (1)
   [junit4]   2> 662341 WARN  (Thread-667) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269362159616 actual=1574170269404102656Retrying with new version .... (1)
   [junit4]   2> 662340 WARN  (Thread-662) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269362159616 actual=1574170269403054080Retrying with new version .... (1)
   [junit4]   2> 662341 WARN  (Thread-660) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269362159616 actual=1574170269404102656Retrying with new version .... (1)
   [junit4]   2> 662341 WARN  (Thread-660) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269404102656 actual=1574170269405151232Retrying with new version .... (2)
   [junit4]   2> 662343 WARN  (Thread-667) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269404102656 actual=1574170269405151232Retrying with new version .... (2)
   [junit4]   2> 662343 WARN  (Thread-667) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269405151232 actual=1574170269407248384Retrying with new version .... (3)
   [junit4]   2> 662343 WARN  (Thread-666) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269403054080 actual=1574170269405151232Retrying with new version .... (2)
   [junit4]   2> 662343 WARN  (Thread-666) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574170269407248384 actual=1574170269407248385Retrying with new version .... (3)
   [junit4]   2> 662356 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1574170269420879872,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 662356 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5c3f69a8 commitCommandVersion:1574170269420879872
   [junit4]   2> 662359 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@412501aa[collection1] main]
   [junit4]   2> 662360 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 662361 INFO  (searcherExecutor-1247-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@412501aa[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2(7.1.0):C10/9:delGen=1)))}
   [junit4]   2> 662361 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 5
   [junit4]   2> 662362 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:10&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662363 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:o88jfs040fgn2zqs60d6+fjijep7wp88ru03xv8qj+dem9yiin4o6ezco3vbtj+0v0tnjyejrkxu2x7fh8x+wkcczv2lvogo4f96ryuc&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 662363 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=int_i:11&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 662384 ERROR (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=int_i:11&wt=xml
   [junit4]   2> 662384 ERROR (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=int_i:11&wt=xml:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=int_i:11&wt=xml
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
   [junit4]   2> 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:264)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 662385 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[AA77EA985CEFFD2E]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testMultipleThreads
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testMultipleThreads -Dtests.seed=AA77EA985CEFFD2E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ja-JP -Dtests.timezone=Chile/Continental -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.09s J0 | AtomicUpdateProcessorFactoryTest.testMultipleThreads <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AA77EA985CEFFD2E:86459D15D02B8BAA]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
   [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:264)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > 	request was:q=int_i:11&wt=xml
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
   [junit4]    > 	... 40 more
   [junit4]   2> 662386 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 662386 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1678785486
   [junit4]   2> 662386 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 662387 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 662388 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 662404 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[AA77EA985CEFFD2E]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 662416 INFO  (coreCloseExecutor-1252-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@657aaef3
   [junit4]   2> 662433 INFO  (coreCloseExecutor-1252-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1702538995
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.update.processor.AtomicUpdateProcessorFactoryTest_AA77EA985CEFFD2E-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {name_s=FST50, title_stringNoNorms=PostingsFormat(name=Memory), title_stemmed=Lucene50(blocksize=128), multiDefault=Lucene50(blocksize=128), cat=PostingsFormat(name=Memory), id=PostingsFormat(name=Memory), text=FST50, title=Lucene50(blocksize=128), title_lettertok=PostingsFormat(name=Memory)}, docValues:{int_i=DocValuesFormat(name=Asserting), range_facet_l_dv=DocValuesFormat(name=Direct), _version_=DocValuesFormat(name=Lucene70), intDefault=DocValuesFormat(name=Lucene70), id_i1=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Lucene70), intDvoDefault=DocValuesFormat(name=Asserting), count_i=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Lucene70), range_facet_l=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1553, maxMBSortInHeap=5.070823918332973, sim=RandomSimilarity(queryNorm=false): {}, locale=ja-JP, timezone=Chile/Continental
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=249842416,total=456654848
   [junit4]   2> NOTE: All tests run in this JVM: [ConnectionManagerTest, LeaderInitiatedRecoveryOnShardRestartTest, XmlUpdateRequestHandlerTest, TestReplicationHandlerBackup, TestExpandComponent, SolrCloudReportersTest, CleanupOldIndexTest, TestNumericTokenStream, DistributedQueueTest, SparseHLLTest, TestBinaryField, TestSolr4Spatial, OverseerRolesTest, HdfsUnloadDistributedZkTest, TestOmitPositions, PeerSyncReplicationTest, TestCloudNestedDocsSort, LeaderFailoverAfterPartitionTest, TestIBSimilarityFactory, BooleanFieldTest, TestSubQueryTransformer, DistributedQueryElevationComponentTest, DeleteReplicaTest, DistributedFacetPivotWhiteBoxTest, TestRTimerTree, TestPKIAuthenticationPlugin, SuggesterWFSTTest, MBeansHandlerTest, TestSolrDeletionPolicy1, TestJavabinTupleStreamParser, TestSolr4Spatial2, TestNamedUpdateProcessors, TestExportWriter, BasicAuthStandaloneTest, BlobRepositoryCloudTest, TestLeaderElectionZkExpiry, TestFunctionQuery, TestSerializedLuceneMatchVersion, RollingRestartTest, BasicDistributedZkTest, AtomicUpdateProcessorFactoryTest]
   [junit4] Completed [135/726 (1!)] on J0 in 1.64s, 4 tests, 1 error <<< FAILURES!

[...truncated 45320 lines...]

[JENKINS] Lucene-Solr-Tests-7.x - Build # 109 - Still unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/109/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderElectionContextKeyTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([BC3F9998ADD9A8F1:346BA6420325C509]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([BC3F9998ADD9A8F1:900DEE15211DDE75]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:260)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=cat:za92uumx5ebsafgax3b5+5igrctj6yi4i2chkuls7+6ynkb9mvtcquc34scqv4+n7m2uf8bsftqkhsfd9nm+ryy9sva4leo7iecz3s8p&wt=xml
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
	... 40 more




Build Log:
[...truncated 11571 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderElectionContextKeyTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/init-core-data-001
   [junit4]   2> 670183 WARN  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 670183 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 670184 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 670185 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 670186 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001
   [junit4]   2> 670186 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 670186 INFO  (Thread-1526) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 670186 INFO  (Thread-1526) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 670198 ERROR (Thread-1526) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 670286 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:51805
   [junit4]   2> 670304 INFO  (jetty-launcher-999-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 670305 INFO  (jetty-launcher-999-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5aacd47{/solr,null,AVAILABLE}
   [junit4]   2> 670311 INFO  (jetty-launcher-999-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@12b833f0{SSL,[ssl, http/1.1]}{127.0.0.1:42733}
   [junit4]   2> 670311 INFO  (jetty-launcher-999-thread-1) [    ] o.e.j.s.Server Started @674974ms
   [junit4]   2> 670311 INFO  (jetty-launcher-999-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42733}
   [junit4]   2> 670311 ERROR (jetty-launcher-999-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 670311 INFO  (jetty-launcher-999-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.1.0
   [junit4]   2> 670311 INFO  (jetty-launcher-999-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 670311 INFO  (jetty-launcher-999-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 670311 INFO  (jetty-launcher-999-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-28T03:25:10.364Z
   [junit4]   2> 670314 INFO  (jetty-launcher-999-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 670324 INFO  (jetty-launcher-999-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51805/solr
   [junit4]   2> 670432 INFO  (jetty-launcher-999-thread-1) [n:127.0.0.1:42733_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 670434 INFO  (jetty-launcher-999-thread-1) [n:127.0.0.1:42733_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42733_solr
   [junit4]   2> 670436 INFO  (jetty-launcher-999-thread-1) [n:127.0.0.1:42733_solr    ] o.a.s.c.Overseer Overseer (id=98383449964085251-127.0.0.1:42733_solr-n_0000000000) starting
   [junit4]   2> 670463 INFO  (jetty-launcher-999-thread-1) [n:127.0.0.1:42733_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42733_solr
   [junit4]   2> 670466 INFO  (zkCallback-1006-thread-1-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 670576 INFO  (jetty-launcher-999-thread-1) [n:127.0.0.1:42733_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42733.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@636ad2cc
   [junit4]   2> 670582 INFO  (jetty-launcher-999-thread-1) [n:127.0.0.1:42733_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42733.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@636ad2cc
   [junit4]   2> 670583 INFO  (jetty-launcher-999-thread-1) [n:127.0.0.1:42733_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42733.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@636ad2cc
   [junit4]   2> 670584 INFO  (jetty-launcher-999-thread-1) [n:127.0.0.1:42733_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/.
   [junit4]   2> 670628 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 670630 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51805/solr ready
   [junit4]   2> 670683 INFO  (qtp1503037063-5924) [n:127.0.0.1:42733_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 670686 INFO  (OverseerThreadFactory-2278-thread-1-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.CreateCollectionCmd Create collection testCollection1
   [junit4]   2> 670808 INFO  (OverseerStateUpdate-98383449964085251-127.0.0.1:42733_solr-n_0000000000) [n:127.0.0.1:42733_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testCollection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:42733/solr",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 670815 INFO  (OverseerStateUpdate-98383449964085251-127.0.0.1:42733_solr-n_0000000000) [n:127.0.0.1:42733_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testCollection1_shard2_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:42733/solr",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 671029 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection1_shard2_replica_n3&action=CREATE&numShards=2&collection=testCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 671030 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 671040 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=testCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 671151 INFO  (zkCallback-1006-thread-1-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 671152 INFO  (zkCallback-1006-thread-2-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 671162 INFO  (zkCallback-1006-thread-3-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 672055 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 672055 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 672063 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.s.IndexSchema [testCollection1_shard2_replica_n3] Schema name=minimal
   [junit4]   2> 672063 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.s.IndexSchema [testCollection1_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 672067 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 672067 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 672067 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testCollection1_shard2_replica_n3' using configuration from collection testCollection1, trusted=true
   [junit4]   2> 672067 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testCollection1_shard1_replica_n1' using configuration from collection testCollection1, trusted=true
   [junit4]   2> 672067 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42733.solr.core.testCollection1.shard1.replica_n1' (registry 'solr.core.testCollection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@636ad2cc
   [junit4]   2> 672067 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42733.solr.core.testCollection1.shard2.replica_n3' (registry 'solr.core.testCollection1.shard2.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@636ad2cc
   [junit4]   2> 672067 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 672068 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 672068 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [[testCollection1_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/testCollection1_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/./testCollection1_shard1_replica_n1/data/]
   [junit4]   2> 672068 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SolrCore [[testCollection1_shard2_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/testCollection1_shard2_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/./testCollection1_shard2_replica_n3/data/]
   [junit4]   2> 672124 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 672124 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 672124 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 672124 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 672126 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 672126 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 672126 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 672126 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 672127 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@33b913de[testCollection1_shard2_replica_n3] main]
   [junit4]   2> 672127 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7ed81b5c[testCollection1_shard1_replica_n1] main]
   [junit4]   2> 672129 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 672129 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 672130 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 672130 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 672130 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 672131 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574135201460649984
   [junit4]   2> 672131 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 672133 INFO  (searcherExecutor-2282-thread-1-processing-n:127.0.0.1:42733_solr x:testCollection1_shard1_replica_n1 s:shard1 c:testCollection1 r:core_node2) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [testCollection1_shard1_replica_n1] Registered new searcher Searcher@7ed81b5c[testCollection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 672133 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574135201462747136
   [junit4]   2> 672142 INFO  (searcherExecutor-2281-thread-1-processing-n:127.0.0.1:42733_solr x:testCollection1_shard2_replica_n3 s:shard2 c:testCollection1 r:core_node4) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SolrCore [testCollection1_shard2_replica_n3] Registered new searcher Searcher@33b913de[testCollection1_shard2_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 672146 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 672146 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 672146 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42733/solr/testCollection1_shard1_replica_n1/
   [junit4]   2> 672146 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 672146 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:42733/solr/testCollection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 672146 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 672147 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 672147 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 672147 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42733/solr/testCollection1_shard2_replica_n3/
   [junit4]   2> 672147 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 672147 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SyncStrategy https://127.0.0.1:42733/solr/testCollection1_shard2_replica_n3/ has no replicas
   [junit4]   2> 672148 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 672151 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42733/solr/testCollection1_shard1_replica_n1/ shard1
   [junit4]   2> 672152 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42733/solr/testCollection1_shard2_replica_n3/ shard2
   [junit4]   2> 672255 INFO  (zkCallback-1006-thread-3-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 672255 INFO  (zkCallback-1006-thread-2-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 672255 INFO  (zkCallback-1006-thread-1-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 672302 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 672303 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 672304 INFO  (qtp1503037063-5925) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=testCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1263
   [junit4]   2> 672306 INFO  (qtp1503037063-5929) [n:127.0.0.1:42733_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection1_shard2_replica_n3&action=CREATE&numShards=2&collection=testCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1276
   [junit4]   2> 672309 INFO  (qtp1503037063-5924) [n:127.0.0.1:42733_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 672411 INFO  (zkCallback-1006-thread-2-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 672411 INFO  (zkCallback-1006-thread-1-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 672411 INFO  (zkCallback-1006-thread-3-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 672689 INFO  (OverseerCollectionConfigSetProcessor-98383449964085251-127.0.0.1:42733_solr-n_0000000000) [n:127.0.0.1:42733_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 673310 INFO  (qtp1503037063-5924) [n:127.0.0.1:42733_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2626
   [junit4]   2> 673311 INFO  (qtp1503037063-5927) [n:127.0.0.1:42733_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 673313 INFO  (OverseerThreadFactory-2278-thread-2-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.CreateCollectionCmd Create collection testCollection2
   [junit4]   2> 673522 INFO  (OverseerStateUpdate-98383449964085251-127.0.0.1:42733_solr-n_0000000000) [n:127.0.0.1:42733_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testCollection2_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:42733/solr",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 673525 INFO  (OverseerStateUpdate-98383449964085251-127.0.0.1:42733_solr-n_0000000000) [n:127.0.0.1:42733_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection2",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testCollection2_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:42733/solr",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 673726 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=testCollection2_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 673727 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection2_shard2_replica_n2&action=CREATE&numShards=2&collection=testCollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 673834 INFO  (zkCallback-1006-thread-1-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 673834 INFO  (zkCallback-1006-thread-3-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 674747 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 674784 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 674792 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.s.IndexSchema [testCollection2_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 674796 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 674796 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testCollection2_shard2_replica_n2' using configuration from collection testCollection2, trusted=true
   [junit4]   2> 674796 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.s.IndexSchema [testCollection2_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 674797 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42733.solr.core.testCollection2.shard2.replica_n2' (registry 'solr.core.testCollection2.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@636ad2cc
   [junit4]   2> 674797 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 674797 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrCore [[testCollection2_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/testCollection2_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/./testCollection2_shard2_replica_n2/data/]
   [junit4]   2> 674800 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 674800 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testCollection2_shard1_replica_n1' using configuration from collection testCollection2, trusted=true
   [junit4]   2> 674802 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42733.solr.core.testCollection2.shard1.replica_n1' (registry 'solr.core.testCollection2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@636ad2cc
   [junit4]   2> 674802 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 674803 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [[testCollection2_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/testCollection2_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001/tempDir-001/node1/./testCollection2_shard1_replica_n1/data/]
   [junit4]   2> 674898 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 674898 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 674900 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 674900 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 674901 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7446bb77[testCollection2_shard2_replica_n2] main]
   [junit4]   2> 674910 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 674911 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 674911 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 674912 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574135204376739840
   [junit4]   2> 674922 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 674922 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 674922 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42733/solr/testCollection2_shard2_replica_n2/
   [junit4]   2> 674922 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 674922 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:42733/solr/testCollection2_shard2_replica_n2/ has no replicas
   [junit4]   2> 674922 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 674927 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42733/solr/testCollection2_shard2_replica_n2/ shard2
   [junit4]   2> 674930 INFO  (searcherExecutor-2291-thread-1-processing-n:127.0.0.1:42733_solr x:testCollection2_shard2_replica_n2 s:shard2 c:testCollection2 r:core_node4) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrCore [testCollection2_shard2_replica_n2] Registered new searcher Searcher@7446bb77[testCollection2_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 674950 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 674950 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 674951 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 674951 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 674953 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@674e0107[testCollection2_shard1_replica_n1] main]
   [junit4]   2> 674963 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 674963 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 674964 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 674964 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574135204431265792
   [junit4]   2> 674968 INFO  (searcherExecutor-2292-thread-1-processing-n:127.0.0.1:42733_solr x:testCollection2_shard1_replica_n1 s:shard1 c:testCollection2 r:core_node3) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [testCollection2_shard1_replica_n1] Registered new searcher Searcher@674e0107[testCollection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 674973 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 674973 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 674973 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42733/solr/testCollection2_shard1_replica_n1/
   [junit4]   2> 674973 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 674973 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:42733/solr/testCollection2_shard1_replica_n1/ has no replicas
   [junit4]   2> 674973 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 674977 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42733/solr/testCollection2_shard1_replica_n1/ shard1
   [junit4]   2> 675079 INFO  (zkCallback-1006-thread-1-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 675080 INFO  (zkCallback-1006-thread-3-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 675127 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 675128 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 675129 INFO  (qtp1503037063-5928) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection2_shard2_replica_n2&action=CREATE&numShards=2&collection=testCollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1402
   [junit4]   2> 675130 INFO  (qtp1503037063-5930) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=testCollection2_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1403
   [junit4]   2> 675133 INFO  (qtp1503037063-5927) [n:127.0.0.1:42733_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 675237 INFO  (zkCallback-1006-thread-1-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 675237 INFO  (zkCallback-1006-thread-3-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 675316 INFO  (OverseerCollectionConfigSetProcessor-98383449964085251-127.0.0.1:42733_solr-n_0000000000) [n:127.0.0.1:42733_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 676133 INFO  (qtp1503037063-5927) [n:127.0.0.1:42733_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2822
   [junit4]   2> 676134 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testCollection1 failOnTimeout:true timeout (sec):30
   [junit4]   2> 676135 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testCollection1
   [junit4]   2> 676135 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testCollection2 failOnTimeout:true timeout (sec):30
   [junit4]   2> 676135 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testCollection2
   [junit4]   2> 676148 INFO  (TEST-LeaderElectionContextKeyTest.test-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 676154 INFO  (qtp1503037063-5924) [n:127.0.0.1:42733_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard2.replica_n3, tag=null
   [junit4]   2> 676158 INFO  (qtp1503037063-5924) [n:127.0.0.1:42733_solr    ] o.a.s.c.SolrCore [testCollection1_shard2_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@397b2e1e
   [junit4]   2> 676161 INFO  (qtp1503037063-5924) [n:127.0.0.1:42733_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard2.replica_n3, tag=964374046
   [junit4]   2> 676161 INFO  (qtp1503037063-5924) [n:127.0.0.1:42733_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection1.shard2.leader, tag=964374046
   [junit4]   2> 676162 INFO  (qtp1503037063-5924) [n:127.0.0.1:42733_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=false&core=testCollection1_shard2_replica_n3&deleteDataDir=false&action=UNLOAD&indexInfo=true&wt=javabin&version=2&deleteIndex=false} status=0 QTime=8
   [junit4]   2> 676163 INFO  (TEST-LeaderElectionContextKeyTest.test-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Ending test
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionContextKeyTest -Dtests.method=test -Dtests.seed=BC3F9998ADD9A8F1 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ms-MY -Dtests.timezone=Etc/UTC -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 0.03s J2 | LeaderElectionContextKeyTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BC3F9998ADD9A8F1:346BA6420325C509]:0)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 676168 INFO  (jetty-closer-1000-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@12b833f0{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 676168 INFO  (jetty-closer-1000-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1095873714
   [junit4]   2> 676169 INFO  (jetty-closer-1000-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 676170 INFO  (jetty-closer-1000-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 676172 INFO  (jetty-closer-1000-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 676173 INFO  (jetty-closer-1000-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 676174 INFO  (coreCloseExecutor-2301-thread-1) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [testCollection1_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4ab98057
   [junit4]   2> 676174 INFO  (coreCloseExecutor-2301-thread-2) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrCore [testCollection2_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@2c982482
   [junit4]   2> 676175 INFO  (coreCloseExecutor-2301-thread-3) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [testCollection2_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7867a273
   [junit4]   2> 676178 INFO  (coreCloseExecutor-2301-thread-1) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard1.replica_n1, tag=1253670999
   [junit4]   2> 676185 INFO  (coreCloseExecutor-2301-thread-1) [n:127.0.0.1:42733_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection1.shard1.leader, tag=1253670999
   [junit4]   2> 676185 INFO  (coreCloseExecutor-2301-thread-2) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard2.replica_n2, tag=748168322
   [junit4]   2> 676188 INFO  (coreCloseExecutor-2301-thread-2) [n:127.0.0.1:42733_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard2.leader, tag=748168322
   [junit4]   2> 676188 INFO  (coreCloseExecutor-2301-thread-3) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard1.replica_n1, tag=2020057715
   [junit4]   2> 676192 INFO  (coreCloseExecutor-2301-thread-3) [n:127.0.0.1:42733_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard1.leader, tag=2020057715
   [junit4]   2> 676192 INFO  (jetty-closer-1000-thread-1) [    ] o.a.s.c.Overseer Overseer (id=98383449964085251-127.0.0.1:42733_solr-n_0000000000) closing
   [junit4]   2> 676193 INFO  (OverseerStateUpdate-98383449964085251-127.0.0.1:42733_solr-n_0000000000) [n:127.0.0.1:42733_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42733_solr
   [junit4]   2> 677694 WARN  (zkCallback-1006-thread-3-processing-n:127.0.0.1:42733_solr) [n:127.0.0.1:42733_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 677694 INFO  (jetty-closer-1000-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5aacd47{/solr,null,UNAVAILABLE}
   [junit4]   2> 677695 ERROR (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 677696 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51805 51805
   [junit4]   2> 678190 INFO  (Thread-1526) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51805 51805
   [junit4]   2> 678191 WARN  (Thread-1526) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	2	/solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	9	/solr/collections/testCollection1/state.json
   [junit4]   2> 	5	/solr/collections/testCollection2/state.json
   [junit4]   2> 	2	/solr/clusterstate.json
   [junit4]   2> 	2	/solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	2	/solr/live_nodes
   [junit4]   2> 	2	/solr/collections
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_BC3F9998ADD9A8F1-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1307, maxMBSortInHeap=5.05627032837331, sim=RandomSimilarity(queryNorm=true): {}, locale=ms-MY, timezone=Etc/UTC
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=158427200,total=526385152
   [junit4]   2> NOTE: All tests run in this JVM: [TestLegacyTerms, TestCloudPivotFacet, TestZkChroot, XsltUpdateRequestHandlerTest, SpellPossibilityIteratorTest, UniqFieldsUpdateProcessorFactoryTest, CoreAdminRequestStatusTest, DistributedDebugComponentTest, DirectoryFactoryTest, TestSmileRequest, ZkStateWriterTest, TestScoreJoinQPNoScore, MoveReplicaTest, HdfsRestartWhileUpdatingTest, SignatureUpdateProcessorFactoryTest, NumericFieldsTest, TestGroupingSearch, TestRawResponseWriter, BlockJoinFacetDistribTest, JSONWriterTest, CircularListTest, BigEndianAscendingWordDeserializerTest, CachingDirectoryFactoryTest, HighlighterMaxOffsetTest, SearchHandlerTest, TestSuggestSpellingConverter, TestSchemaVersionResource, ConfigSetsAPITest, SharedFSAutoReplicaFailoverUtilsTest, TestTestInjection, SolrCloudExampleTest, CurrencyFieldTypeTest, TestPayloadScoreQParserPlugin, TestDocTermOrds, TestSolrCloudWithHadoopAuthPlugin, SuggesterTest, TestReversedWildcardFilterFactory, TestHdfsBackupRestoreCore, TestTolerantSearch, TestExactSharedStatsCache, TestPivotHelperCode, HdfsNNFailoverTest, TestManagedStopFilterFactory, TestHighFrequencyDictionaryFactory, TestRequestStatusCollectionAPI, CdcrReplicationDistributedZkTest, AutoCommitTest, TestRestoreCore, CdcrVersionReplicationTest, RequiredFieldsTest, SortSpecParsingTest, SimplePostToolTest, TestObjectReleaseTracker, TestSolrCloudWithKerberosAlt, DistributedQueryComponentOptimizationTest, TestOnReconnectListenerSupport, RulesTest, BufferStoreTest, TestLMJelinekMercerSimilarityFactory, RemoteQueryErrorTest, AliasIntegrationTest, TestSolrFieldCacheBean, MinimalSchemaTest, DirectUpdateHandlerTest, CollectionReloadTest, TestStressInPlaceUpdates, V2ApiIntegrationTest, TestSimpleTextCodec, SuggesterFSTTest, TestShortCircuitedRequests, OverseerModifyCollectionTest, ShowFileRequestHandlerTest, BasicAuthIntegrationTest, TestSha256AuthenticationProvider, SearchHandlerTest, CloudMLTQParserTest, CustomCollectionTest, SimpleFacetsTest, PolyFieldTest, BadIndexSchemaTest, LeaderElectionContextKeyTest]
   [junit4] Completed [209/726 (1!)] on J2 in 8.02s, 1 test, 1 failure <<< FAILURES!

[...truncated 1627 lines...]
   [junit4] Suite: org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.update.processor.AtomicUpdateProcessorFactoryTest_BC3F9998ADD9A8F1-001/init-core-data-001
   [junit4]   2> 2450512 WARN  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=17 numCloses=17
   [junit4]   2> 2450512 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 2450513 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2450514 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2450514 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2450514 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2450546 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 2450577 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2450711 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2450756 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b73bce6
   [junit4]   2> 2450762 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b73bce6
   [junit4]   2> 2450762 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b73bce6
   [junit4]   2> 2450773 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2450810 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 2450838 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2450981 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2450994 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 2450994 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@b73bce6
   [junit4]   2> 2450995 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2450995 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.update.processor.AtomicUpdateProcessorFactoryTest_BC3F9998ADD9A8F1-001/init-core-data-001/]
   [junit4]   2> 2450998 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=40, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7843335318143118]
   [junit4]   2> 2451074 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2451074 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2451075 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2451075 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2451076 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=69.3583984375, floorSegmentMB=0.8173828125, forceMergeDeletesPctAllowed=4.661645847058243, segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5303341861835681
   [junit4]   2> 2451077 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4220e21d[collection1] main]
   [junit4]   2> 2451077 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2451086 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2451115 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2451115 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2451115 INFO  (coreLoadExecutor-6457-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574137064877719552
   [junit4]   2> 2451115 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2451115 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2451115 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2451115 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2451116 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2451116 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2451116 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2451116 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2451117 INFO  (searcherExecutor-6458-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4220e21d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2451119 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2451122 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasics
   [junit4]   2> 2451126 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1574137064889253888,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2451126 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2a6a1a45 commitCommandVersion:1574137064889253888
   [junit4]   2> 2451130 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@c3bd50[collection1] main]
   [junit4]   2> 2451131 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2451133 INFO  (searcherExecutor-6458-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@c3bd50[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C1)))}
   [junit4]   2> 2451133 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 7
   [junit4]   2> 2451136 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:1&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451137 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:human&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451137 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=title:Mr&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451138 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=count_i:20&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451138 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name_s:Virat&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451139 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=multiDefault:Delhi&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451141 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1574137064904982528,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2451141 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2a6a1a45 commitCommandVersion:1574137064904982528
   [junit4]   2> 2451145 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@2394e586[collection1] main]
   [junit4]   2> 2451145 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2451147 INFO  (searcherExecutor-6458-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2394e586[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(7.1.0):C1)))}
   [junit4]   2> 2451148 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 6
   [junit4]   2> 2451148 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:1&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451149 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:human&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451149 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:animal&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451150 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=title:Mr&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 2451150 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=title:Dr&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451150 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=count_i:20&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 2451151 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=count_i:40&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451151 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name_s:Virat&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 2451151 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=multiDefault:Delhi&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 2451151 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testBasics-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBasics
   [junit4]   2> 2451154 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testNoUniqueIdPassed-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNoUniqueIdPassed
   [junit4]   2> 2451154 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testNoUniqueIdPassed-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNoUniqueIdPassed
   [junit4]   2> 2451157 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMultipleThreads
   [junit4]   2> 2451157 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 2451157 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@368266e5[collection1] realtime]
   [junit4]   2> 2451158 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
   [junit4]   2> 2451161 WARN  (Thread-5889) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574137064923856896 actual=1574137064924905472Retrying with new version .... (1)
   [junit4]   2> 2451177 WARN  (Thread-5891) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574137064933294080 actual=1574137064941682688Retrying with new version .... (1)
   [junit4]   2> 2451177 WARN  (Thread-5892) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574137064925954050 actual=1574137064941682688Retrying with new version .... (1)
   [junit4]   2> 2451178 WARN  (Thread-5893) [    ] o.a.s.u.p.AtomicUpdateProcessorFactory Atomic update failed due to version conflict for 10 expected=1574137064933294080 actual=1574137064941682688Retrying with new version .... (1)
   [junit4]   2> 2451179 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1574137064944828416,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2451179 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2a6a1a45 commitCommandVersion:1574137064944828416
   [junit4]   2> 2451183 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@22488d68[collection1] main]
   [junit4]   2> 2451185 INFO  (searcherExecutor-6458-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@22488d68[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2(7.1.0):C10/9:delGen=1)))}
   [junit4]   2> 2451185 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2451186 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 6
   [junit4]   2> 2451186 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:10&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 2451187 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=cat:za92uumx5ebsafgax3b5+5igrctj6yi4i2chkuls7+6ynkb9mvtcquc34scqv4+n7m2uf8bsftqkhsfd9nm+ryy9sva4leo7iecz3s8p&wt=xml} hits=0 status=0 QTime=0
   [junit4]   2> 2451188 ERROR (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=cat:za92uumx5ebsafgax3b5+5igrctj6yi4i2chkuls7+6ynkb9mvtcquc34scqv4+n7m2uf8bsftqkhsfd9nm+ryy9sva4leo7iecz3s8p&wt=xml
   [junit4]   2> 2451188 ERROR (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=cat:za92uumx5ebsafgax3b5+5igrctj6yi4i2chkuls7+6ynkb9mvtcquc34scqv4+n7m2uf8bsftqkhsfd9nm+ryy9sva4leo7iecz3s8p&wt=xml:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=cat:za92uumx5ebsafgax3b5+5igrctj6yi4i2chkuls7+6ynkb9mvtcquc34scqv4+n7m2uf8bsftqkhsfd9nm+ryy9sva4leo7iecz3s8p&wt=xml
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
   [junit4]   2> 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:260)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$Statement

[...truncated too long message...]

ules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 2451189 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testMultipleThreads-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testMultipleThreads
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testMultipleThreads -Dtests.seed=BC3F9998ADD9A8F1 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=en-GB -Dtests.timezone=Australia/Queensland -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.04s J0 | AtomicUpdateProcessorFactoryTest.testMultipleThreads <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BC3F9998ADD9A8F1:900DEE15211DDE75]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
   [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:260)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > 	request was:q=cat:za92uumx5ebsafgax3b5+5igrctj6yi4i2chkuls7+6ynkb9mvtcquc34scqv4+n7m2uf8bsftqkhsfd9nm+ryy9sva4leo7iecz3s8p&wt=xml
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
   [junit4]    > 	... 40 more
   [junit4]   2> 2451197 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testWrongAtomicOpPassed-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testWrongAtomicOpPassed
   [junit4]   2> 2451197 INFO  (TEST-AtomicUpdateProcessorFactoryTest.testWrongAtomicOpPassed-seed#[BC3F9998ADD9A8F1]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testWrongAtomicOpPassed
   [junit4]   2> 2451197 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2451197 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1786732673
   [junit4]   2> 2451197 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 2451199 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 2451200 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 2451200 INFO  (SUITE-AtomicUpdateProcessorFactoryTest-seed#[BC3F9998ADD9A8F1]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2451200 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@542a5dc1
   [junit4]   2> 2451202 INFO  (coreCloseExecutor-6463-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1412062657
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.update.processor.AtomicUpdateProcessorFactoryTest_BC3F9998ADD9A8F1-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {name_s=FST50, title_stringNoNorms=PostingsFormat(name=LuceneFixedGap), title_stemmed=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), cat=PostingsFormat(name=LuceneFixedGap), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), text=FST50, title=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), title_lettertok=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Asserting), _version_=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Direct), intDvoDefault=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1338, maxMBSortInHeap=5.016175830271302, sim=RandomSimilarity(queryNorm=false): {}, locale=en-GB, timezone=Australia/Queensland
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=306237056,total=529530880
   [junit4]   2> NOTE: All tests run in this JVM: [TestLMDirichletSimilarityFactory, DeleteReplicaTest, DistributedSuggestComponentTest, SolrPluginUtilsTest, TestDocumentBuilder, SmileWriterTest, CdcrUpdateLogTest, PeerSyncTest, TestBadConfig, ConfigureRecoveryStrategyTest, TestLeaderElectionZkExpiry, LeaderInitiatedRecoveryOnShardRestartTest, OutputWriterTest, TestAddFieldRealTimeGet, WordBreakSolrSpellCheckerTest, TestFieldTypeCollectionResource, SuggestComponentContextFilterQueryTest, SolrCoreTest, LeaderFailoverAfterPartitionTest, SolrMetricReporterTest, TestRandomDVFaceting, TestSolr4Spatial2, TestStandardQParsers, FullHLLTest, ConnectionReuseTest, TestTolerantUpdateProcessorRandomCloud, ReplaceNodeTest, TestSSLRandomization, TestMultiValuedNumericRangeQuery, BasicZkTest, TestUninvertingReader, SparseHLLTest, HdfsChaosMonkeySafeLeaderTest, CollectionsAPISolrJTest, TestSQLHandler, TestCloudPseudoReturnFields, TestDeleteCollectionOnDownNodes, PeerSyncReplicationTest, TestHighlightDedupGrouping, TestExactStatsCache, TestMacroExpander, TestRTimerTree, DistributedQueueTest, ForceLeaderTest, DistributedSpellCheckComponentTest, HdfsRecoverLeaseTest, TestJavabinTupleStreamParser, TestSolrQueryParser, TestManagedSynonymFilterFactory, ConnectionManagerTest, CdcrReplicationHandlerTest, TestManagedResource, OverseerRolesTest, VMParamsZkACLAndCredentialsProvidersTest, TestRandomFlRTGCloud, ZkNodePropsTest, SolrIndexSplitterTest, MultiThreadedOCPTest, TestStressLiveNodes, HttpSolrCallGetCoreTest, DistributedExpandComponentTest, TestPointFields, TestCloudInspectUtil, TestStressLucene, UUIDFieldTest, TestInfoStreamLogging, TestConfigSetsAPIExclusivity, BlobRepositoryCloudTest, TestBlobHandler, RAMDirectoryFactoryTest, TestSchemaResource, HighlighterConfigTest, RollingRestartTest, TestSerializedLuceneMatchVersion, FieldMutatingUpdateProcessorTest, LeaderElectionIntegrationTest, OpenExchangeRatesOrgProviderTest, TestFaceting, TestSolrDeletionPolicy1, TestUseDocValuesAsStored, TestRangeQuery, TestFilteredDocIdSet, ClassificationUpdateProcessorIntegrationTest, TestPhraseSuggestions, TestRawTransformer, TestIBSimilarityFactory, TestWordDelimiterFilterFactory, TestNamedUpdateProcessors, ChaosMonkeyNothingIsSafeTest, TestComplexPhraseQParserPlugin, CoreMergeIndexesAdminHandlerTest, DistribCursorPagingTest, EchoParamsTest, TestNonDefinedSimilarityFactory, CleanupOldIndexTest, SpellCheckCollatorWithCollapseTest, CollectionStateFormat2Test, BooleanFieldTest, MetricsConfigTest, BasicDistributedZkTest, SpatialRPTFieldTypeTest, TestConfigsApi, TestCollationField, TestExpandComponent, DistributedFacetPivotWhiteBoxTest, TestPrepRecovery, TestUnifiedSolrHighlighter, TestFileDictionaryLookup, ExitableDirectoryReaderTest, TestExclusionRuleCollectionAccess, TestSearcherReuse, DistanceUnitsTest, TestDynamicFieldResource, TestSolrConfigHandler, TestRealTimeGet, TestOverriddenPrefixQueryForCustomFieldType, CursorPagingTest, SolrCloudReportersTest, TestSolrCloudSnapshots, TestSystemCollAutoCreate, DistributedFacetPivotLargeTest, ReplicaListTransformerTest, ShufflingReplicaListTransformerTest, SpatialHeatmapFacetsTest, TestNumericRangeQuery64, SolrMetricManagerTest, SolrMetricsIntegrationTest, SolrGangliaReporterTest, SolrSlf4jReporterTest, TestGraphMLResponseWriter, TestPushWriter, TestSolrQueryResponse, TestChildDocTransformer, TestSubQueryTransformerDistrib, TestCloudManagedSchema, TestSchemaManager, AnalyticsQueryTest, LargeFieldTest, RankQueryTest, TestCollapseQParserPlugin, TestCustomSort, TestFieldSortValues, TestHashQParserPlugin, TestLegacyNumericRangeQueryBuilder, TestMaxScoreQueryParser, TestNoOpRegenerator, TestPayloadCheckQParserPlugin, TestRecoveryHdfs, BasicAuthStandaloneTest, PKIAuthenticationIntegrationTest, TestImpersonationWithHadoopAuth, BlockCacheTest, HdfsDirectoryTest, UpdateLogTest, VersionInfoTest, AddSchemaFieldsUpdateProcessorFactoryTest, AtomicUpdateProcessorFactoryTest]
   [junit4] Completed [708/726 (2!)] on J0 in 0.71s, 4 tests, 1 error <<< FAILURES!

[...truncated 43349 lines...]

[JENKINS] Lucene-Solr-Tests-7.x - Build # 108 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/108/

2 tests failed.
FAILED:  org.apache.solr.update.TestInPlaceUpdatesDistrib.test

Error Message:
inplace_updatable_float didn't match for replica at client: http://127.0.0.1:41516/vl_/dv/collection1 expected:<102.0> but was:<101.0>

Stack Trace:
java.lang.AssertionError: inplace_updatable_float didn't match for replica at client: http://127.0.0.1:41516/vl_/dv/collection1 expected:<102.0> but was:<101.0>
	at __randomizedtesting.SeedInfo.seed([BBBB3F832A2FA6BC:33EF005984D3CB44]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.apache.solr.update.TestInPlaceUpdatesDistrib.assertReplicaValue(TestInPlaceUpdatesDistrib.java:980)
	at org.apache.solr.update.TestInPlaceUpdatesDistrib.delayedReorderingFetchesMissingUpdateFromLeaderTest(TestInPlaceUpdatesDistrib.java:875)
	at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([BBBB3F832A2FA6BC:9789480EA6EBD038]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:264)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=int_i:22&wt=xml
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
	... 40 more




Build Log:
[...truncated 11010 lines...]
   [junit4] Suite: org.apache.solr.update.TestInPlaceUpdatesDistrib
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/init-core-data-001
   [junit4]   2> 451054 WARN  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 451054 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 451067 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 451067 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 451068 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /vl_/dv
   [junit4]   2> 451070 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 451070 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 451199 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 451382 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=inplace-updates
   [junit4]   2> 451401 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 451511 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 451527 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 451528 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 451529 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 451619 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 451673 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=inplace-updates
   [junit4]   2> 451676 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 451676 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 451692 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 451693 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 451693 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/init-core-data-001/]
   [junit4]   2> 451718 WARN  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 451885 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 451885 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 451886 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 451886 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 451903 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@51067141[collection1] main]
   [junit4]   2> 451904 WARN  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 451904 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 451905 INFO  (searcherExecutor-534-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@51067141[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 451905 INFO  (coreLoadExecutor-533-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574109719481548800
   [junit4]   2> 451907 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[BBBB3F832A2FA6BC]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 451950 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 451951 INFO  (Thread-1680) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 451951 INFO  (Thread-1680) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 451986 ERROR (Thread-1680) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 452058 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.ZkTestServer start zk server on port:33140
   [junit4]   2> 452221 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 452222 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/schema-inplace-updates.xml to /configs/conf1/schema.xml
   [junit4]   2> 452222 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 452223 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 452224 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 452237 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 452257 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 452258 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 452258 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 452259 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 452260 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 452277 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15d85c732e10001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 452277 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
   [junit4]   2> 452747 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 452749 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c44b58a{/vl_/dv,null,AVAILABLE}
   [junit4]   2> 452749 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6db313f0{HTTP/1.1,[http/1.1]}{127.0.0.1:35282}
   [junit4]   2> 452749 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.Server Started @462886ms
   [junit4]   2> 452749 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/tempDir-001/control/data, hostContext=/vl_/dv, hostPort=35282, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/control-001/cores}
   [junit4]   2> 452750 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 452750 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 452750 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 452750 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 452750 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-27T20:40:11.520Z
   [junit4]   2> 452792 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 452792 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/control-001/solr.xml
   [junit4]   2> 452796 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 452815 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33140/solr
   [junit4]   2> 453077 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 453077 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35282_vl_%2Fdv
   [junit4]   2> 453078 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.Overseer Overseer (id=98381857470611460-127.0.0.1:35282_vl_%2Fdv-n_0000000000) starting
   [junit4]   2> 453150 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35282_vl_%2Fdv
   [junit4]   2> 453151 INFO  (zkCallback-218-thread-1-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 453696 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 453839 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 453839 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 453840 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/control-001/cores
   [junit4]   2> 454111 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 454111 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33140/solr ready
   [junit4]   2> 454149 INFO  (qtp1879758873-2667) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:35282_vl_%252Fdv&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 454220 INFO  (OverseerThreadFactory-544-thread-1-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 454403 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 454403 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 454549 INFO  (zkCallback-218-thread-1-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 455556 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 455601 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=inplace-updates
   [junit4]   2> 455604 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 455604 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 455604 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 455605 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 455605 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 455629 WARN  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 455743 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 455743 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 455744 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 455744 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 455745 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@bd2715f[control_collection_shard1_replica_n1] main]
   [junit4]   2> 455760 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 455760 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 455761 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 455761 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574109723524857856
   [junit4]   2> 455788 INFO  (searcherExecutor-547-thread-1-processing-n:127.0.0.1:35282_vl_%2Fdv x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@bd2715f[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 455806 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 455806 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 455806 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35282/vl_/dv/control_collection_shard1_replica_n1/
   [junit4]   2> 455807 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 455807 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:35282/vl_/dv/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 455807 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 455809 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35282/vl_/dv/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 455924 INFO  (zkCallback-218-thread-1-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 455972 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 455974 INFO  (qtp1879758873-2665) [n:127.0.0.1:35282_vl_%2Fdv c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1571
   [junit4]   2> 456001 INFO  (qtp1879758873-2667) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 456107 INFO  (zkCallback-218-thread-1-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 456248 INFO  (OverseerCollectionConfigSetProcessor-98381857470611460-127.0.0.1:35282_vl_%2Fdv-n_0000000000) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 457002 INFO  (qtp1879758873-2667) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:35282_vl_%252Fdv&wt=javabin&version=2} status=0 QTime=2852
   [junit4]   2> 457015 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15d85c732e10005, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 457112 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 457181 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33140/solr ready
   [junit4]   2> 457181 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 457183 INFO  (qtp1879758873-2671) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 457226 INFO  (OverseerThreadFactory-544-thread-2-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 457234 WARN  (OverseerThreadFactory-544-thread-2-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 457532 INFO  (qtp1879758873-2671) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 457532 INFO  (qtp1879758873-2671) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=349
   [junit4]   2> 458483 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-1-001 of type TLOG
   [junit4]   2> 458484 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 458684 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1fe5f856{/vl_/dv,null,AVAILABLE}
   [junit4]   2> 458684 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@17565983{HTTP/1.1,[http/1.1]}{127.0.0.1:41516}
   [junit4]   2> 458685 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.Server Started @468821ms
   [junit4]   2> 458685 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/vl_/dv, hostPort=41516, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-1-001/cores}
   [junit4]   2> 458685 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 458685 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 458685 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 458685 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 458685 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-27T20:40:17.455Z
   [junit4]   2> 458721 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 458721 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-1-001/solr.xml
   [junit4]   2> 458737 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 458793 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33140/solr
   [junit4]   2> 458884 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 458886 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 458940 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41516_vl_%2Fdv
   [junit4]   2> 458941 INFO  (zkCallback-225-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 458955 INFO  (zkCallback-218-thread-2-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 459004 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 459263 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 459284 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 459296 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 459297 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-1-001/cores
   [junit4]   2> 459341 INFO  (OverseerCollectionConfigSetProcessor-98381857470611460-127.0.0.1:35282_vl_%2Fdv-n_0000000000) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 459476 INFO  (qtp2026158121-2709) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:41516_vl_%252Fdv&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 459505 INFO  (OverseerThreadFactory-544-thread-3-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:41516_vl_%2Fdv for creating new replica
   [junit4]   2> 459524 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 459525 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 459564 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 460604 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 460680 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.s.IndexSchema [collection1_shard1_replica_t1] Schema name=inplace-updates
   [junit4]   2> 460695 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 460695 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t1' using configuration from collection collection1, trusted=true
   [junit4]   2> 460696 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t1' (registry 'solr.core.collection1.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 460696 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 460696 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.SolrCore [[collection1_shard1_replica_t1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-1-001/cores/collection1_shard1_replica_t1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-1-001/cores/collection1_shard1_replica_t1/data/]
   [junit4]   2> 460768 WARN  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 461060 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 461060 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 461192 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 461192 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 461193 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@11da9efd[collection1_shard1_replica_t1] main]
   [junit4]   2> 461194 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 461194 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 461194 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 461194 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574109729221771264
   [junit4]   2> 461196 INFO  (searcherExecutor-558-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv x:collection1_shard1_replica_t1 s:shard1 c:collection1) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.SolrCore [collection1_shard1_replica_t1] Registered new searcher Searcher@11da9efd[collection1_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 461211 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 461211 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 461211 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41516/vl_/dv/collection1_shard1_replica_t1/
   [junit4]   2> 461211 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 461211 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.SyncStrategy http://127.0.0.1:41516/vl_/dv/collection1_shard1_replica_t1/ has no replicas
   [junit4]   2> 461211 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 461211 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.ZkController collection1_shard1_replica_t1 stopping background replication from leader
   [junit4]   2> 461230 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41516/vl_/dv/collection1_shard1_replica_t1/ shard1
   [junit4]   2> 461339 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 461383 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 461385 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1861
   [junit4]   2> 461429 INFO  (qtp2026158121-2709) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41516_vl_%252Fdv&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1952
   [junit4]   2> 461501 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 461514 INFO  (OverseerCollectionConfigSetProcessor-98381857470611460-127.0.0.1:35282_vl_%2Fdv-n_0000000000) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 461920 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-2-001 of type TLOG
   [junit4]   2> 461933 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 461934 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3eede08b{/vl_/dv,null,AVAILABLE}
   [junit4]   2> 461934 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@552eeb7b{HTTP/1.1,[http/1.1]}{127.0.0.1:45031}
   [junit4]   2> 461934 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.Server Started @472071ms
   [junit4]   2> 461934 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/vl_/dv, hostPort=45031, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-2-001/cores}
   [junit4]   2> 461935 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 461935 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 461935 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 461935 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 461935 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-27T20:40:20.705Z
   [junit4]   2> 461995 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 461995 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-2-001/solr.xml
   [junit4]   2> 462012 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 462026 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33140/solr
   [junit4]   2> 462069 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 462084 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 462085 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45031_vl_%2Fdv
   [junit4]   2> 462107 INFO  (zkCallback-218-thread-2-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 462107 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 462108 INFO  (zkCallback-236-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 462108 INFO  (zkCallback-225-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 462410 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 462444 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 462444 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 462445 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-2-001/cores
   [junit4]   2> 462727 INFO  (qtp1947092621-2738) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45031_vl_%252Fdv&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 462777 INFO  (OverseerThreadFactory-544-thread-4-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:45031_vl_%2Fdv for creating new replica
   [junit4]   2> 462785 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 462786 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 462959 INFO  (zkCallback-236-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 462963 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 463896 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 463970 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.s.IndexSchema [collection1_shard1_replica_t3] Schema name=inplace-updates
   [junit4]   2> 463986 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 463986 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t3' using configuration from collection collection1, trusted=true
   [junit4]   2> 463986 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t3' (registry 'solr.core.collection1.shard1.replica_t3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 463986 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 463986 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.c.SolrCore [[collection1_shard1_replica_t3] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-2-001/cores/collection1_shard1_replica_t3], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-2-001/cores/collection1_shard1_replica_t3/data/]
   [junit4]   2> 464009 WARN  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 464453 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 464453 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 464454 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 464454 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 464455 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.s.SolrIndexSearcher Opening [Searcher@259b3d5d[collection1_shard1_replica_t3] main]
   [junit4]   2> 464457 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 464457 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 464490 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 464491 INFO  (searcherExecutor-569-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.c.SolrCore [collection1_shard1_replica_t3] Registered new searcher Searcher@259b3d5d[collection1_shard1_replica_t3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 464491 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574109732678926336
   [junit4]   2> 464528 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_t3
   [junit4]   2> 464547 INFO  (qtp1947092621-2736) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1  x:collection1_shard1_replica_t3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1761
   [junit4]   2> 464551 INFO  (updateExecutor-233-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 464553 INFO  (qtp1947092621-2738) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45031_vl_%252Fdv&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1826
   [junit4]   2> 464571 INFO  (OverseerCollectionConfigSetProcessor-98381857470611460-127.0.0.1:35282_vl_%2Fdv-n_0000000000) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 464607 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 464660 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 464660 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.ZkController collection1_shard1_replica_t3 stopping background replication from leader
   [junit4]   2> 464660 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_t3]
   [junit4]   2> 464660 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 464660 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_t3] as recovering, leader is [http://127.0.0.1:41516/vl_/dv/collection1_shard1_replica_t1/] and I am [http://127.0.0.1:45031/vl_/dv/collection1_shard1_replica_t3/]
   [junit4]   2> 464749 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:41516/vl_/dv]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_t1&nodeName=127.0.0.1:45031_vl_%252Fdv&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 464751 INFO  (qtp2026158121-2712) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 464771 INFO  (zkCallback-236-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 464782 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 464786 INFO  (qtp2026158121-2712) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_t1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:45031_vl_%2Fdv, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1_shard1_replica_t3","base_url":"http://127.0.0.1:45031/vl_/dv","node_name":"127.0.0.1:45031_vl_%2Fdv","state":"down","type":"TLOG"}
   [junit4]   2> 465173 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-3-001 of type TLOG
   [junit4]   2> 465174 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 465265 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@779b0b93{/vl_/dv,null,AVAILABLE}
   [junit4]   2> 465265 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@fa78130{HTTP/1.1,[http/1.1]}{127.0.0.1:36168}
   [junit4]   2> 465266 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.e.j.s.Server Started @475402ms
   [junit4]   2> 465266 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/tempDir-001/jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/vl_/dv, hostPort=36168, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-3-001/cores}
   [junit4]   2> 465266 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 465285 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 465285 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 465285 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 465285 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-27T20:40:24.055Z
   [junit4]   2> 465339 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 465339 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-3-001/solr.xml
   [junit4]   2> 465376 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 465411 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33140/solr
   [junit4]   2> 465602 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 465604 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 465655 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36168_vl_%2Fdv
   [junit4]   2> 465656 INFO  (zkCallback-236-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 465658 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 465657 INFO  (zkCallback-225-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 465678 INFO  (zkCallback-218-thread-2-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 465738 INFO  (zkCallback-243-thread-1-processing-n:127.0.0.1:36168_vl_%2Fdv) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 465788 INFO  (qtp2026158121-2712) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_t1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:45031_vl_%2Fdv, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1_shard1_replica_t3","base_url":"http://127.0.0.1:45031/vl_/dv","node_name":"127.0.0.1:45031_vl_%2Fdv","state":"recovering","type":"TLOG"}
   [junit4]   2> 465788 INFO  (qtp2026158121-2712) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 465788 INFO  (qtp2026158121-2712) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:45031_vl_%252Fdv&onlyIfLeaderActive=true&core=collection1_shard1_replica_t1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1037
   [junit4]   2> 466183 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 466203 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 466203 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@794c0e0f
   [junit4]   2> 466204 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[BBBB3F832A2FA6BC]) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_BBBB3F832A2FA6BC-001/shard-3-001/cores
   [junit4]   2> 466373 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
   [junit4]   2> 466373 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:41516/vl_/dv/collection1_shard1_replica_t1/].
   [junit4]   2> 466395 INFO  (qtp2026158121-2706) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1574109734675415040,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 466478 INFO  (qtp2026158121-2706) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 466734 INFO  (qtp1947092621-2739) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:36168_vl_%252Fdv&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 466742 INFO  (qtp2026158121-2706) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 466742 INFO  (qtp2026158121-2706) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_t1]  webapp=/vl_/dv path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2}{commit=} 0 347
   [junit4]   2> 466752 INFO  (OverseerThreadFactory-544-thread-5-processing-n:127.0.0.1:35282_vl_%2Fdv) [n:127.0.0.1:35282_vl_%2Fdv    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:36168_vl_%2Fdv for creating new replica
   [junit4]   2> 466877 INFO  (qtp319743511-2771) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t5&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 466877 INFO  (qtp319743511-2771) [n:127.0.0.1:36168_vl_%2Fdv    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 466894 INFO  (qtp2026158121-2711) [n:127.0.0.1:41516_vl_%2Fdv c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1]  webapp=/vl_/dv path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1
   [junit4]   2> 466894 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 466894 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 466894 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 466894 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 466895 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Replication Recovery was successful.
   [junit4]   2> 466895 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 466895 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.ZkController collection1_shard1_replica_t3 starting background replication from leader
   [junit4]   2> 466895 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 466934 INFO  (zkCallback-236-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv) [n:127.0.0.1:45031_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 466949 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 466949 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 467058 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery.
   [junit4]   2> 467058 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574109735370620928
   [junit4]   2> 467095 INFO  (recoveryExecutor-234-thread-1-processing-n:127.0.0.1:45031_vl_%2Fdv x:collection1_shard1_replica_t3 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:45031_vl_%2Fdv c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t3] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true]
   [junit4]   2> 467253 INFO  (zkCallback-230-thread-1-processing-n:127.0.0.1:41516_vl_%2Fdv) [n:127.0.0.1:41516_vl_%2Fdv    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 467277 INFO  (zkC

[...truncated too long message...]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'tvWindowStream' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 9. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/request/SimpleFacets.java (at line 912)
 [ecj-lint] 	fastForRandomSet = new HashDocSet(sset.getDocs(), 0, sset.size());
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'fastForRandomSet' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 10. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/response/SmileResponseWriter.java (at line 33)
 [ecj-lint] 	new SmileWriter(out, request, response).writeResponse();
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 11. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 146)
 [ecj-lint] 	ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 12. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/CollapsingQParserPlugin.java (at line 1315)
 [ecj-lint] 	UninvertingReader uninvertingReader = new UninvertingReader(new ReaderWrapper(searcher.getSlowAtomicReader(), collapseField), mapping);
 [ecj-lint] 	                  ^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'uninvertingReader' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 13. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 145)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] 14. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 154)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] 15. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 163)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 16. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 78)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 17. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 78)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 18. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java (at line 287)
 [ecj-lint] 	throw new SolrException(ErrorCode.SERVER_ERROR, "Error occurred while loading solr.xml from zookeeper", e);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'loader' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 19. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java (at line 98)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 20. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 191)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 21. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 261)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 22. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 228)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 23. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 228)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 24. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 260)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 25. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 275)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 26. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 529)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 27. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UUIDUpdateProcessorFactory.java (at line 23)
 [ecj-lint] 	import org.apache.solr.common.SolrException;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.SolrException is never used
 [ecj-lint] ----------
 [ecj-lint] 28. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UUIDUpdateProcessorFactory.java (at line 24)
 [ecj-lint] 	import static org.apache.solr.common.SolrException.ErrorCode.*;
 [ecj-lint] 	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.SolrException.ErrorCode is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 29. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessor.java (at line 85)
 [ecj-lint] 	UpdateRequestProcessor p = this;
 [ecj-lint] 	                       ^
 [ecj-lint] Resource leak: 'p' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 30. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] 31. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 32. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SolrLogLayout.java (at line 149)
 [ecj-lint] 	SolrQueryRequest req = requestInfo == null ? null : requestInfo.getReq();
 [ecj-lint] 	                 ^^^
 [ecj-lint] Resource leak: 'req' is never closed
 [ecj-lint] ----------
 [ecj-lint] 32 problems (2 errors, 30 warnings)

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:810: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:101: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:689: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2007: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2046: Compile failed; see the compiler error output for details.

Total time: 161 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-Tests-7.x - Build # 107 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/107/

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderElectionContextKeyTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([D79E61F761ACD88F:5FCA5E2DCF50B577]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 10950 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderElectionContextKeyTest
   [junit4]   2> 174463 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/init-core-data-001
   [junit4]   2> 174464 WARN  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
   [junit4]   2> 174464 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 174466 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 174466 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001
   [junit4]   2> 174466 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 174467 INFO  (Thread-250) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 174467 INFO  (Thread-250) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 174469 ERROR (Thread-250) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 174567 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:44725
   [junit4]   2> 174578 INFO  (jetty-launcher-217-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 174579 INFO  (jetty-launcher-217-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@49fd267c{/solr,null,AVAILABLE}
   [junit4]   2> 174581 INFO  (jetty-launcher-217-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1b20db92{HTTP/1.1,[http/1.1]}{127.0.0.1:40276}
   [junit4]   2> 174581 INFO  (jetty-launcher-217-thread-1) [    ] o.e.j.s.Server Started @177421ms
   [junit4]   2> 174581 INFO  (jetty-launcher-217-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40276}
   [junit4]   2> 174582 ERROR (jetty-launcher-217-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 174582 INFO  (jetty-launcher-217-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.1.0
   [junit4]   2> 174582 INFO  (jetty-launcher-217-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 174584 INFO  (jetty-launcher-217-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 174584 INFO  (jetty-launcher-217-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-27T15:34:37.611Z
   [junit4]   2> 174588 INFO  (jetty-launcher-217-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 174598 INFO  (jetty-launcher-217-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44725/solr
   [junit4]   2> 174739 INFO  (jetty-launcher-217-thread-1) [n:127.0.0.1:40276_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 174740 INFO  (jetty-launcher-217-thread-1) [n:127.0.0.1:40276_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40276_solr
   [junit4]   2> 174741 INFO  (jetty-launcher-217-thread-1) [n:127.0.0.1:40276_solr    ] o.a.s.c.Overseer Overseer (id=98380655984312323-127.0.0.1:40276_solr-n_0000000000) starting
   [junit4]   2> 174753 INFO  (jetty-launcher-217-thread-1) [n:127.0.0.1:40276_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40276_solr
   [junit4]   2> 174755 INFO  (zkCallback-224-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 174822 INFO  (jetty-launcher-217-thread-1) [n:127.0.0.1:40276_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40276.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39905d43
   [junit4]   2> 174828 INFO  (jetty-launcher-217-thread-1) [n:127.0.0.1:40276_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40276.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39905d43
   [junit4]   2> 174828 INFO  (jetty-launcher-217-thread-1) [n:127.0.0.1:40276_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40276.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39905d43
   [junit4]   2> 174829 INFO  (jetty-launcher-217-thread-1) [n:127.0.0.1:40276_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/.
   [junit4]   2> 174860 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 174862 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44725/solr ready
   [junit4]   2> 174874 INFO  (qtp191249480-1273) [n:127.0.0.1:40276_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 174880 INFO  (OverseerThreadFactory-605-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.CreateCollectionCmd Create collection testCollection1
   [junit4]   2> 174990 INFO  (OverseerStateUpdate-98380655984312323-127.0.0.1:40276_solr-n_0000000000) [n:127.0.0.1:40276_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testCollection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:40276/solr",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 174994 INFO  (OverseerStateUpdate-98380655984312323-127.0.0.1:40276_solr-n_0000000000) [n:127.0.0.1:40276_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testCollection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:40276/solr",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 175201 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection1_shard2_replica_n2&action=CREATE&numShards=2&collection=testCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 175201 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=testCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 175201 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 175313 INFO  (zkCallback-224-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 175314 INFO  (zkCallback-224-thread-2-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 176224 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 176226 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 176234 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.s.IndexSchema [testCollection1_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 176234 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.s.IndexSchema [testCollection1_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 176237 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 176237 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 176237 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testCollection1_shard2_replica_n2' using configuration from collection testCollection1, trusted=true
   [junit4]   2> 176237 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testCollection1_shard1_replica_n1' using configuration from collection testCollection1, trusted=true
   [junit4]   2> 176244 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40276.solr.core.testCollection1.shard2.replica_n2' (registry 'solr.core.testCollection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39905d43
   [junit4]   2> 176245 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 176245 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SolrCore [[testCollection1_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/testCollection1_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/./testCollection1_shard2_replica_n2/data/]
   [junit4]   2> 176259 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40276.solr.core.testCollection1.shard1.replica_n1' (registry 'solr.core.testCollection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39905d43
   [junit4]   2> 176259 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 176259 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [[testCollection1_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/testCollection1_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/./testCollection1_shard1_replica_n1/data/]
   [junit4]   2> 176326 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 176327 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 176341 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 176342 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 176343 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 176343 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 176343 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 176343 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 176345 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6e784e30[testCollection1_shard1_replica_n1] main]
   [junit4]   2> 176348 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 176348 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@143053ed[testCollection1_shard2_replica_n2] main]
   [junit4]   2> 176349 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 176350 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 176350 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 176351 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 176351 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574090497723465728
   [junit4]   2> 176354 INFO  (searcherExecutor-609-thread-1-processing-n:127.0.0.1:40276_solr x:testCollection1_shard2_replica_n2 s:shard2 c:testCollection1 r:core_node4) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SolrCore [testCollection1_shard2_replica_n2] Registered new searcher Searcher@143053ed[testCollection1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 176355 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 176356 INFO  (searcherExecutor-608-thread-1-processing-n:127.0.0.1:40276_solr x:testCollection1_shard1_replica_n1 s:shard1 c:testCollection1 r:core_node3) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [testCollection1_shard1_replica_n1] Registered new searcher Searcher@6e784e30[testCollection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 176357 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574090497729757184
   [junit4]   2> 176364 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 176364 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 176364 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40276/solr/testCollection1_shard2_replica_n2/
   [junit4]   2> 176364 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 176364 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40276/solr/testCollection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 176364 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 176374 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 176374 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 176374 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40276/solr/testCollection1_shard1_replica_n1/
   [junit4]   2> 176374 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 176374 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:40276/solr/testCollection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 176374 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 176378 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40276/solr/testCollection1_shard2_replica_n2/ shard2
   [junit4]   2> 176386 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40276/solr/testCollection1_shard1_replica_n1/ shard1
   [junit4]   2> 176492 INFO  (zkCallback-224-thread-2-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 176492 INFO  (zkCallback-224-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 176529 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 176531 INFO  (qtp191249480-1277) [n:127.0.0.1:40276_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection1_shard2_replica_n2&action=CREATE&numShards=2&collection=testCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1330
   [junit4]   2> 176540 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 176541 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=testCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1340
   [junit4]   2> 176552 INFO  (qtp191249480-1273) [n:127.0.0.1:40276_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 176645 INFO  (zkCallback-224-thread-2-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 176645 INFO  (zkCallback-224-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 176883 INFO  (OverseerCollectionConfigSetProcessor-98380655984312323-127.0.0.1:40276_solr-n_0000000000) [n:127.0.0.1:40276_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 177553 INFO  (qtp191249480-1273) [n:127.0.0.1:40276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2678
   [junit4]   2> 177554 INFO  (qtp191249480-1274) [n:127.0.0.1:40276_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 177559 INFO  (OverseerThreadFactory-605-thread-2-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.CreateCollectionCmd Create collection testCollection2
   [junit4]   2> 177773 INFO  (OverseerStateUpdate-98380655984312323-127.0.0.1:40276_solr-n_0000000000) [n:127.0.0.1:40276_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testCollection2_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:40276/solr",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 177779 INFO  (OverseerStateUpdate-98380655984312323-127.0.0.1:40276_solr-n_0000000000) [n:127.0.0.1:40276_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection2",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testCollection2_shard2_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:40276/solr",
   [junit4]   2>   "type":"NRT"} 
   [junit4]   2> 177982 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection2_shard2_replica_n3&action=CREATE&numShards=2&collection=testCollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 177982 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=testCollection2_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 178094 INFO  (zkCallback-224-thread-2-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 178094 INFO  (zkCallback-224-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 179016 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 179021 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 179031 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.s.IndexSchema [testCollection2_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 179035 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.s.IndexSchema [testCollection2_shard2_replica_n3] Schema name=minimal
   [junit4]   2> 179038 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 179038 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testCollection2_shard2_replica_n3' using configuration from collection testCollection2, trusted=true
   [junit4]   2> 179039 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40276.solr.core.testCollection2.shard2.replica_n3' (registry 'solr.core.testCollection2.shard2.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39905d43
   [junit4]   2> 179039 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 179039 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore [[testCollection2_shard2_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/testCollection2_shard2_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/./testCollection2_shard2_replica_n3/data/]
   [junit4]   2> 179045 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 179045 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testCollection2_shard1_replica_n1' using configuration from collection testCollection2, trusted=true
   [junit4]   2> 179047 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40276.solr.core.testCollection2.shard1.replica_n1' (registry 'solr.core.testCollection2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39905d43
   [junit4]   2> 179047 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 179047 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [[testCollection2_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/testCollection2_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001/tempDir-001/node1/./testCollection2_shard1_replica_n1/data/]
   [junit4]   2> 179118 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 179118 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 179120 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 179120 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 179122 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@303176e6[testCollection2_shard1_replica_n1] main]
   [junit4]   2> 179127 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 179127 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 179128 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 179129 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574090500636409856
   [junit4]   2> 179129 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 179129 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 179131 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 179131 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 179137 INFO  (searcherExecutor-619-thread-1-processing-n:127.0.0.1:40276_solr x:testCollection2_shard1_replica_n1 s:shard1 c:testCollection2 r:core_node2) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [testCollection2_shard1_replica_n1] Registered new searcher Searcher@303176e6[testCollection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 179138 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@58018b6e[testCollection2_shard2_replica_n3] main]
   [junit4]   2> 179138 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 179138 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 179138 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40276/solr/testCollection2_shard1_replica_n1/
   [junit4]   2> 179138 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 179138 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:40276/solr/testCollection2_shard1_replica_n1/ has no replicas
   [junit4]   2> 179139 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 179139 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 179140 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 179141 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 179141 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1574090500648992768
   [junit4]   2> 179144 INFO  (searcherExecutor-618-thread-1-processing-n:127.0.0.1:40276_solr x:testCollection2_shard2_replica_n3 s:shard2 c:testCollection2 r:core_node4) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore [testCollection2_shard2_replica_n3] Registered new searcher Searcher@58018b6e[testCollection2_shard2_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 179150 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 179150 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 179150 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40276/solr/testCollection2_shard2_replica_n3/
   [junit4]   2> 179150 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 179150 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SyncStrategy http://127.0.0.1:40276/solr/testCollection2_shard2_replica_n3/ has no replicas
   [junit4]   2> 179150 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 179153 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40276/solr/testCollection2_shard1_replica_n1/ shard1
   [junit4]   2> 179161 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40276/solr/testCollection2_shard2_replica_n3/ shard2
   [junit4]   2> 179264 INFO  (zkCallback-224-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 179264 INFO  (zkCallback-224-thread-2-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 179305 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 179307 INFO  (qtp191249480-1270) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=testCollection2_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1324
   [junit4]   2> 179313 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 179314 INFO  (qtp191249480-1275) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection2_shard2_replica_n3&action=CREATE&numShards=2&collection=testCollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1331
   [junit4]   2> 179317 INFO  (qtp191249480-1274) [n:127.0.0.1:40276_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 179417 INFO  (zkCallback-224-thread-2-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 179421 INFO  (zkCallback-224-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 179561 INFO  (OverseerCollectionConfigSetProcessor-98380655984312323-127.0.0.1:40276_solr-n_0000000000) [n:127.0.0.1:40276_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 180317 INFO  (qtp191249480-1274) [n:127.0.0.1:40276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2763
   [junit4]   2> 180318 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testCollection1 failOnTimeout:true timeout (sec):30
   [junit4]   2> 180318 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testCollection1
   [junit4]   2> 180318 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testCollection2 failOnTimeout:true timeout (sec):30
   [junit4]   2> 180319 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testCollection2
   [junit4]   2> 180332 INFO  (TEST-LeaderElectionContextKeyTest.test-seed#[D79E61F761ACD88F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 180338 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard2.replica_n2, tag=null
   [junit4]   2> 180342 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr    ] o.a.s.c.SolrCore [testCollection1_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@5347dc8e
   [junit4]   2> 180345 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard2.replica_n2, tag=1397218446
   [junit4]   2> 180345 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection1.shard2.leader, tag=1397218446
   [junit4]   2> 180352 INFO  (qtp191249480-1276) [n:127.0.0.1:40276_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=false&core=testCollection1_shard2_replica_n2&deleteDataDir=false&action=UNLOAD&indexInfo=true&wt=javabin&version=2&deleteIndex=false} status=0 QTime=14
   [junit4]   2> 180354 INFO  (TEST-LeaderElectionContextKeyTest.test-seed#[D79E61F761ACD88F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending test
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionContextKeyTest -Dtests.method=test -Dtests.seed=D79E61F761ACD88F -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=vi -Dtests.timezone=Europe/Vienna -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.04s J2 | LeaderElectionContextKeyTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D79E61F761ACD88F:5FCA5E2DCF50B577]:0)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 180365 INFO  (jetty-closer-218-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1b20db92{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 180366 INFO  (jetty-closer-218-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=160810774
   [junit4]   2> 180366 INFO  (jetty-closer-218-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 180368 INFO  (jetty-closer-218-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 180369 INFO  (jetty-closer-218-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 180371 INFO  (jetty-closer-218-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 180373 INFO  (coreCloseExecutor-628-thread-2) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [testCollection2_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1fcfa4e6
   [junit4]   2> 180373 INFO  (coreCloseExecutor-628-thread-1) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [testCollection1_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1f3e13a3
   [junit4]   2> 180373 INFO  (coreCloseExecutor-628-thread-3) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore [testCollection2_shard2_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@8cf7605
   [junit4]   2> 180382 INFO  (coreCloseExecutor-628-thread-2) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard1.replica_n1, tag=533701862
   [junit4]   2> 180385 INFO  (coreCloseExecutor-628-thread-2) [n:127.0.0.1:40276_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard1.leader, tag=533701862
   [junit4]   2> 180385 INFO  (coreCloseExecutor-628-thread-3) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard2.replica_n3, tag=147813893
   [junit4]   2> 180389 INFO  (coreCloseExecutor-628-thread-3) [n:127.0.0.1:40276_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard2.leader, tag=147813893
   [junit4]   2> 180389 INFO  (coreCloseExecutor-628-thread-1) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard1.replica_n1, tag=524161955
   [junit4]   2> 180393 INFO  (coreCloseExecutor-628-thread-1) [n:127.0.0.1:40276_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection1.shard1.leader, tag=524161955
   [junit4]   2> 180394 INFO  (jetty-closer-218-thread-1) [    ] o.a.s.c.Overseer Overseer (id=98380655984312323-127.0.0.1:40276_solr-n_0000000000) closing
   [junit4]   2> 180394 INFO  (OverseerStateUpdate-98380655984312323-127.0.0.1:40276_solr-n_0000000000) [n:127.0.0.1:40276_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40276_solr
   [junit4]   2> 181895 WARN  (zkCallback-224-thread-1-processing-n:127.0.0.1:40276_solr) [n:127.0.0.1:40276_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 181896 INFO  (jetty-closer-218-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@49fd267c{/solr,null,UNAVAILABLE}
   [junit4]   2> 181897 ERROR (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 181897 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[D79E61F761ACD88F]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44725 44725
   [junit4]   2> 182069 INFO  (Thread-250) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44725 44725
   [junit4]   2> 182070 WARN  (Thread-250) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	2	/solr/aliases.json
   [junit4]   2> 	2	/solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	5	/solr/collections/testCollection2/state.json
   [junit4]   2> 	5	/solr/collections/testCollection1/state.json
   [junit4]   2> 	2	/solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	2	/solr/live_nodes
   [junit4]   2> 	2	/solr/collections
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderElectionContextKeyTest_D79E61F761ACD88F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1564, maxMBSortInHeap=6.650041070611806, sim=RandomSimilarity(queryNorm=true): {}, locale=vi, timezone=Europe/Vienna
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=142576392,total=428343296
   [junit4]   2> NOTE: All tests run in this JVM: [MissingSegmentRecoveryTest, TestFoldingMultitermQuery, SimplePostToolTest, TestCloudPivotFacet, SaslZkACLProviderTest, AddSchemaFieldsUpdateProcessorFactoryTest, RemoteQueryErrorTest, TestFieldCacheSort, SearchHandlerTest, TestDFRSimilarityFactory, DirectoryFactoryTest, TestHighFrequencyDictionaryFactory, TestSort, TestTestInjection, PathHierarchyTokenizerFactoryTest, DistributedDebugComponentTest, HdfsChaosMonkeyNothingIsSafeTest, JSONWriterTest, CopyFieldTest, BufferStoreTest, TestStressReorder, TestTolerantUpdateProcessorCloud, TestSubQueryTransformerDistrib, TestIndexSearcher, TermsComponentTest, LeaderElectionContextKeyTest]
   [junit4] Completed [53/726 (1!)] on J2 in 7.62s, 1 test, 1 failure <<< FAILURES!

[...truncated 41988 lines...]
-ecj-javadoc-lint-src:
    [mkdir] Created dir: /tmp/ecj270021846
 [ecj-lint] Compiling 1118 source files to /tmp/ecj270021846
 [ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
 [ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
 [ecj-lint] ----------
 [ecj-lint] 1. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/core/CoreContainer.java (at line 972)
 [ecj-lint] 	core = new SolrCore(this, dcore, coreConfig);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'core' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 2. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/core/HdfsDirectoryFactory.java (at line 234)
 [ecj-lint] 	dir = new BlockDirectory(path, hdfsDir, cache, null, blockCacheReadEnabled, false, cacheMerges, cacheReadOnce);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'dir' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 3. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 120)
 [ecj-lint] 	reader = cfiltfac.create(reader);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'reader' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] 4. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 144)
 [ecj-lint] 	return namedList;
 [ecj-lint] 	^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'listBasedTokenStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 5. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java (at line 1282)
 [ecj-lint] 	DirectoryReader reader = s==null ? null : s.get().getIndexReader();
 [ecj-lint] 	                ^^^^^^
 [ecj-lint] Resource leak: 'reader' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 6. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/handler/sql/SolrTable.java (at line 517)
 [ecj-lint] 	ParallelStream parallelStream = new ParallelStream(zk, collection, tupleStream, numWorkers, comp);
 [ecj-lint] 	               ^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'parallelStream' is never closed
 [ecj-lint] ----------
 [ecj-lint] 7. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/handler/sql/SolrTable.java (at line 743)
 [ecj-lint] 	ParallelStream parallelStream = new ParallelStream(zkHost, collection, tupleStream, numWorkers, comp);
 [ecj-lint] 	               ^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'parallelStream' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 8. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java (at line 578)
 [ecj-lint] 	tvWindowStream = new OffsetWindowTokenFilter(tvStream);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'tvWindowStream' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 9. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/request/SimpleFacets.java (at line 912)
 [ecj-lint] 	fastForRandomSet = new HashDocSet(sset.getDocs(), 0, sset.size());
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'fastForRandomSet' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 10. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/response/SmileResponseWriter.java (at line 33)
 [ecj-lint] 	new SmileWriter(out, request, response).writeResponse();
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 11. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 146)
 [ecj-lint] 	ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 12. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/CollapsingQParserPlugin.java (at line 1315)
 [ecj-lint] 	UninvertingReader uninvertingReader = new UninvertingReader(new ReaderWrapper(searcher.getSlowAtomicReader(), collapseField), mapping);
 [ecj-lint] 	                  ^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'uninvertingReader' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 13. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 145)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] 14. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 154)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] 15. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 163)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 16. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 78)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 17. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 78)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 18. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java (at line 287)
 [ecj-lint] 	throw new SolrException(ErrorCode.SERVER_ERROR, "Error occurred while loading solr.xml from zookeeper", e);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'loader' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 19. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java (at line 98)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 20. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 191)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 21. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 261)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 22. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 228)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 23. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 228)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 24. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 260)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 25. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 275)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 26. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 529)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 27. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UUIDUpdateProcessorFactory.java (at line 23)
 [ecj-lint] 	import org.apache.solr.common.SolrException;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.SolrException is never used
 [ecj-lint] ----------
 [ecj-lint] 28. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UUIDUpdateProcessorFactory.java (at line 24)
 [ecj-lint] 	import static org.apache.solr.common.SolrException.ErrorCode.*;
 [ecj-lint] 	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.SolrException.ErrorCode is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 29. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessor.java (at line 85)
 [ecj-lint] 	UpdateRequestProcessor p = this;
 [ecj-lint] 	                       ^
 [ecj-lint] Resource leak: 'p' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 30. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] 31. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 32. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SolrLogLayout.java (at line 149)
 [ecj-lint] 	SolrQueryRequest req = requestInfo == null ? null : requestInfo.getReq();
 [ecj-lint] 	                 ^^^
 [ecj-lint] Resource leak: 'req' is never closed
 [ecj-lint] ----------
 [ecj-lint] 32 problems (2 errors, 30 warnings)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:810: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:101: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:689: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2007: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2046: Compile failed; see the compiler error output for details.

Total time: 85 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-Tests-7.x - Build # 106 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/106/

4 tests failed.
FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:44290

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:44290
	at __randomizedtesting.SeedInfo.seed([2689BFDD1C91E35C:FEC4928AEB4C46FC]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:637)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:417)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:334)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:983)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:523)
	... 44 more


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestDynamicLoading

Error Message:
75 threads leaked from SUITE scope at org.apache.solr.core.TestDynamicLoading:     1) Thread[id=5126, name=qtp1582688695-5126-acceptor-0@40c7eb50-ServerConnector@1d37324c{SSL,[ssl, http/1.1]}{127.0.0.1:44741}, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=5130, name=qtp1582688695-5130, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=5184, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=5256, name=zkCallback-589-thread-1-processing-n:127.0.0.1:44741_-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)    5) Thread[id=5270, name=zkCallback-601-thread-6, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=5125, name=qtp1582688695-5125, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=5158, name=qtp529031077-5158, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=5190, name=qtp621988132-5190-acceptor-0@5a4e7cd7-ServerConnector@1c874efe{SSL,[ssl, http/1.1]}{127.0.0.1:44324}, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    9) Thread[id=5191, name=qtp621988132-5191, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   10) Thread[id=5153, name=qtp529031077-5153, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   11) Thread[id=5152, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   12) Thread[id=5154, name=qtp529031077-5154, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   13) Thread[id=5137, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   14) Thread[id=5223, name=zkCallback-595-thread-4, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   15) Thread[id=5195, name=qtp621988132-5195, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   16) Thread[id=5150, name=searcherExecutor-2035-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   17) Thread[id=5211, name=Scheduler-1327792833, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   18) Thread[id=5224, name=zkCallback-595-thread-5, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   19) Thread[id=5159, name=qtp529031077-5159, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   20) Thread[id=5161, name=org.eclipse.jetty.server.session.HashSessionManager@36ef00b0Timer, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   21) Thread[id=5144, name=Thread-1301, state=WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1591)   22) Thread[id=5233, name=zkCallback-601-thread-5-processing-n:127.0.0.1:44324_-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   23) Thread[id=5255, name=zkCallback-589-thread-1-processing-n:127.0.0.1:44741_-SendThread(127.0.0.1:44520), state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)   24) Thread[id=5218, name=zkCallback-589-thread-3, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   25) Thread[id=5271, name=zkCallback-601-thread-7, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   26) Thread[id=5155, name=qtp529031077-5155-acceptor-0@39ee9d51-ServerConnector@6d386b6e{SSL,[ssl, http/1.1]}{127.0.0.1:36179}, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   27) Thread[id=5131, name=qtp1582688695-5131, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   28) Thread[id=5188, name=qtp621988132-5188, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   29) Thread[id=5220, name=zkCallback-589-thread-5, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   30) Thread[id=5182, name=searcherExecutor-2046-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   31) Thread[id=5156, name=qtp529031077-5156, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   32) Thread[id=5207, name=zkCallback-601-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   33) Thread[id=5127, name=qtp1582688695-5127, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   34) Thread[id=5128, name=qtp1582688695-5128, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   35) Thread[id=5208, name=Thread-1319, state=WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1591)   36) Thread[id=5272, name=zkCallback-589-thread-7, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   37) Thread[id=5243, name=OverseerHdfsCoreFailoverThread-98379230584438802-127.0.0.1:44324_-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)         at java.lang.Thread.run(Thread.java:748)   38) Thread[id=5132, name=org.eclipse.jetty.server.session.HashSessionManager@203c049aTimer, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   39) Thread[id=5194, name=qtp621988132-5194, state=RUNNABLE, group=TGRP-TestDynamicLoading]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   40) Thread[id=5240, name=zkCallback-595-thread-2-processing-n:127.0.0.1:36179_-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   41) Thread[id=5177, name=Scheduler-667082715, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   42) Thread[id=5231, name=zkCallback-601-thread-5, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   43) Thread[id=5239, name=zkCallback-595-thread-2-processing-n:127.0.0.1:36179_-SendThread(127.0.0.1:44520), state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)   44) Thread[id=5143, name=zkCallback-589-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   45) Thread[id=5242, name=OverseerCollectionConfigSetProcessor-98379230584438802-127.0.0.1:44324_-n_0000000004, state=TIMED_WAITING, group=Overseer collection creation process.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)         at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:387)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:193)         at java.lang.Thread.run(Thread.java:748)   46) Thread[id=5172, name=zkCallback-595-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   47) Thread[id=5196, name=org.eclipse.jetty.server.session.HashSessionManager@6715f634Timer, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   48) Thread[id=5227, name=zkCallback-601-thread-4, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   49) Thread[id=5225, name=zkCallback-601-thread-2, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   50) Thread[id=5232, name=zkCallback-601-thread-5-processing-n:127.0.0.1:44324_-SendThread(127.0.0.1:44520), state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)   51) Thread[id=5192, name=qtp621988132-5192, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   52) Thread[id=5129, name=qtp1582688695-5129, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   53) Thread[id=5219, name=zkCallback-589-thread-4, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   54) Thread[id=5157, name=qtp529031077-5157, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   55) Thread[id=5136, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   56) Thread[id=5165, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   57) Thread[id=5238, name=zkCallback-595-thread-6, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   58) Thread[id=5262, name=searcherExecutor-2064-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   59) Thread[id=5193, name=qtp621988132-5193, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   60) Thread[id=5221, name=zkCallback-595-thread-2, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   61) Thread[id=5160, name=qtp529031077-5160, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   62) Thread[id=5200, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   63) Thread[id=5222, name=zkCallback-595-thread-3, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   64) Thread[id=5250, name=OverseerThreadFactory-2063-thread-1, state=WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   65) Thread[id=5124, name=qtp1582688695-5124, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   66) Thread[id=5147, name=Scheduler-360315604, state=WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   67) Thread[id=5166, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   68) Thread[id=5173, name=Thread-1310, state=WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1591)   69) Thread[id=5226, name=zkCallback-601-thread-3, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   70) Thread[id=5217, name=zkCallback-589-thread-2, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   71) Thread[id=5254, name=zkCallback-589-thread-6, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   72) Thread[id=5189, name=qtp621988132-5189, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   73) Thread[id=5201, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   74) Thread[id=5273, name=zkCallback-595-thread-7, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   75) Thread[id=5241, name=OverseerStateUpdate-98379230584438802-127.0.0.1:44324_-n_0000000004, state=TIMED_WAITING, group=Overseer state updater.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:165)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:139)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:184)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 75 threads leaked from SUITE scope at org.apache.solr.core.TestDynamicLoading: 
   1) Thread[id=5126, name=qtp1582688695-5126-acceptor-0@40c7eb50-ServerConnector@1d37324c{SSL,[ssl, http/1.1]}{127.0.0.1:44741}, state=RUNNABLE, group=TGRP-TestDynamicLoading]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=5130, name=qtp1582688695-5130, state=RUNNABLE, group=TGRP-TestDynamicLoading]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=5184, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=5256, name=zkCallback-589-thread-1-processing-n:127.0.0.1:44741_-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
   5) Thread[id=5270, name=zkCallback-601-thread-6, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=5125, name=qtp1582688695-5125, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=5158, name=qtp529031077-5158, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   8) Thread[id=5190, name=qtp621988132-5190-acceptor-0@5a4e7cd7-ServerConnector@1c874efe{SSL,[ssl, http/1.1]}{127.0.0.1:44324}, state=RUNNABLE, group=TGRP-TestDynamicLoading]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   9) Thread[id=5191, name=qtp621988132-5191, state=RUNNABLE, group=TGRP-TestDynamicLoading]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  10) Thread[id=5153, name=qtp529031077-5153, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  11) Thread[id=5152, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  12) Thread[id=5154, name=qtp529031077-5154, state=RUNNABLE, group=TGRP-TestDynamicLoading]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  13) Thread[id=5137, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  14) Thread[id=5223, name=zkCallback-595-thread-4, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  15) Thread[id=5195, name=qtp621988132-5195, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  16) Thread[id=5150, name=searcherExecutor-2035-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  17) Thread[id=5211, name=Scheduler-1327792833, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  18) Thread[id=5224, name=zkCallback-595-thread-5, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  19) Thread[id=5159, name=qtp529031077-5159, state=RUNNABLE, group=TGRP-TestDynamicLoading]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  20) Thread[id=5161, name=org.eclipse.jetty.server.session.HashSessionManager@36ef00b0Timer, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  21) Thread[id=5144, name=Thread-1301, state=WAITING, group=TGRP-TestDynamicLoading]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1591)
  22) Thread[id=5233, name=zkCallback-601-thread-5-processing-n:127.0.0.1:44324_-EventThread, state=WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
  23) Thread[id=5255, name=zkCallback-589-thread-1-processing-n:127.0.0.1:44741_-SendThread(127.0.0.1:44520), state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
  24) Thread[id=5218, name=zkCallback-589-thread-3, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  25) Thread[id=5271, name=zkCallback-601-thread-7, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  26) Thread[id=5155, name=qtp529031077-5155-acceptor-0@39ee9d51-ServerConnector@6d386b6e{SSL,[ssl, http/1.1]}{127.0.0.1:36179}, state=RUNNABLE, group=TGRP-TestDynamicLoading]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  27) Thread[id=5131, name=qtp1582688695-5131, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  28) Thread[id=5188, name=qtp621988132-5188, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  29) Thread[id=5220, name=zkCallback-589-thread-5, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  30) Thread[id=5182, name=searcherExecutor-2046-thread-1, state=WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  31) Thread[id=5156, name=qtp529031077-5156, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  32) Thread[id=5207, name=zkCallback-601-thread-1, state=TIMED_WAITING, group=TGRP-TestDynamicLoading]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  33) Thread[id=5127, name=qtp1582688695-5127, state=RUNNABLE, group=TGRP-TestDynamicLoading]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  34) Th

[...truncated too long message...]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'tvWindowStream' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 9. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/request/SimpleFacets.java (at line 912)
 [ecj-lint] 	fastForRandomSet = new HashDocSet(sset.getDocs(), 0, sset.size());
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'fastForRandomSet' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 10. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/response/SmileResponseWriter.java (at line 33)
 [ecj-lint] 	new SmileWriter(out, request, response).writeResponse();
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 11. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 146)
 [ecj-lint] 	ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 12. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/CollapsingQParserPlugin.java (at line 1315)
 [ecj-lint] 	UninvertingReader uninvertingReader = new UninvertingReader(new ReaderWrapper(searcher.getSlowAtomicReader(), collapseField), mapping);
 [ecj-lint] 	                  ^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'uninvertingReader' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 13. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 145)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] 14. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 154)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] 15. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 163)
 [ecj-lint] 	HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint] 	           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 16. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 78)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 17. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 78)
 [ecj-lint] 	SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint] 	                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 18. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java (at line 287)
 [ecj-lint] 	throw new SolrException(ErrorCode.SERVER_ERROR, "Error occurred while loading solr.xml from zookeeper", e);
 [ecj-lint] 	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'loader' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 19. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java (at line 98)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 20. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 191)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 21. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 261)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 22. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 228)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 23. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 228)
 [ecj-lint] 	ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint] 	                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 24. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 260)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 25. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 275)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 26. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 529)
 [ecj-lint] 	LogCodec codec = new LogCodec(resolver);
 [ecj-lint] 	         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 27. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UUIDUpdateProcessorFactory.java (at line 23)
 [ecj-lint] 	import org.apache.solr.common.SolrException;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.SolrException is never used
 [ecj-lint] ----------
 [ecj-lint] 28. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UUIDUpdateProcessorFactory.java (at line 24)
 [ecj-lint] 	import static org.apache.solr.common.SolrException.ErrorCode.*;
 [ecj-lint] 	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.SolrException.ErrorCode is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 29. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessor.java (at line 85)
 [ecj-lint] 	UpdateRequestProcessor p = this;
 [ecj-lint] 	                       ^
 [ecj-lint] Resource leak: 'p' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 30. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] 31. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
 [ecj-lint] 	BAOS bos =  new BAOS();
 [ecj-lint] 	     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 32. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/java/org/apache/solr/util/SolrLogLayout.java (at line 149)
 [ecj-lint] 	SolrQueryRequest req = requestInfo == null ? null : requestInfo.getReq();
 [ecj-lint] 	                 ^^^
 [ecj-lint] Resource leak: 'req' is never closed
 [ecj-lint] ----------
 [ecj-lint] 32 problems (2 errors, 30 warnings)

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:810: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:101: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:689: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2007: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2046: Compile failed; see the compiler error output for details.

Total time: 133 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any