You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2019/04/23 14:08:32 UTC

[JENKINS] Lucene-Solr-8.x-MacOSX (64bit/jdk-9) - Build # 97 - Still Failing!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/97/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 13781 lines...]
   [junit4] JVM J0: stdout was not empty, see: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20190423_124430_5086724720299292625791.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] #
   [junit4] # A fatal error has been detected by the Java Runtime Environment:
   [junit4] #
   [junit4] #  Internal Error (sharedRuntime.cpp:931), pid=43509, tid=28951
   [junit4] #  guarantee(cm != NULL) failed: must have containing compiled method for implicit division-by-zero exceptions
   [junit4] #
   [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0+181) (build 9+181)
   [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (9+181, mixed mode, tiered, compressed oops, serial gc, bsd-amd64)
   [junit4] # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again[thread 46867 also had an error]
   [junit4] 
   [junit4] #
   [junit4] [thread 20995 also had an error]
   [junit4] # An error report file with more information is saved as:
   [junit4] # /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/hs_err_pid43509.log
   [junit4] #
   [junit4] # If you would like to submit a bug report, please visit:
   [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
   [junit4] #
   [junit4] <<< JVM J0: EOF ----

[...truncated 1817 lines...]
   [junit4] ERROR: JVM J0 ended with an exception, command line: /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/bin/java -XX:+UseCompressedOops -XX:+UseSerialGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/heapdumps -ea -esa --illegal-access=deny -Dtests.prefix=tests -Dtests.seed=511CC36784EE7551 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=8.1.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=1 -DtempDir=./temp -Djava.io.tmpdir=./temp -Dcommon.dir=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene -Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/clover/db -Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=8.1.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.src.home=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX -Djava.security.egd=file:/dev/./urandom -Djunit4.childvm.cwd=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0 -Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/temp -Djunit4.childvm.id=0 -Djunit4.childvm.count=2 -Dfile.encoding=US-ASCII -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dtests.filterstacks=true -Dtests.leaveTemporary=false -Dtests.badapples=false -classpath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/classes/test:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-test-framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/test-framework/lib/hamcrest-core-1.3.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/test-framework/lib/junit4-ant-2.7.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/test-framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-solrj/classes/java:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/classes/java:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/analysis/common/lucene-analyzers-common-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/analysis/nori/lucene-analyzers-nori-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/codecs/lucene-codecs-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/backward-codecs/lucene-backward-codecs-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/highlighter/lucene-highlighter-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/memory/lucene-memory-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/misc/lucene-misc-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/spatial-extras/lucene-spatial-extras-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/spatial3d/lucene-spatial3d-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/expressions/lucene-expressions-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/suggest/lucene-suggest-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/grouping/lucene-grouping-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/queries/lucene-queries-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/queryparser/lucene-queryparser-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/join/lucene-join-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/sandbox/lucene-sandbox-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/classification/lucene-classification-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/antlr4-runtime-4.5.1-1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/asm-5.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/asm-commons-5.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/avatica-core-1.13.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/caffeine-2.4.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/calcite-core-1.18.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/calcite-linq4j-1.18.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-beanutils-1.9.3.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-cli-1.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-codec-1.11.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-collections-3.2.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-compiler-3.0.9.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-configuration2-2.1.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-exec-1.3.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-fileupload-1.3.3.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-io-2.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-lang3-3.8.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-math3-3.6.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/commons-text-1.6.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/curator-client-2.13.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/curator-framework-2.13.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/curator-recipes-2.13.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/disruptor-3.4.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/eigenbase-properties-1.1.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/guava-25.1-jre.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/hadoop-annotations-3.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/hadoop-auth-3.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/hadoop-common-3.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/hadoop-hdfs-client-3.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/hppc-0.8.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/htrace-core4-4.1.0-incubating.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/jackson-annotations-2.9.8.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/jackson-core-2.9.8.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/jackson-databind-2.9.8.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/jackson-dataformat-smile-2.9.8.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/janino-3.0.9.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/jcl-over-slf4j-1.7.24.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/jose4j-0.6.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/json-path-2.4.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/kerb-core-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/kerb-util-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/kerby-asn1-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/kerby-pkix-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/log4j-1.2-api-2.11.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/log4j-api-2.11.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/log4j-core-2.11.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/log4j-slf4j-impl-2.11.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/org.restlet-2.3.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/protobuf-java-3.6.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/re2j-1.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/rrd4j-3.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/spatial4j-0.7.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/lib/t-digest-3.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/commons-io-2.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/commons-math3-3.6.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/http2-client-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/http2-common-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/http2-hpack-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/http2-http-client-transport-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/httpclient-4.5.6.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/httpcore-4.4.10.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/httpmime-4.5.6.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/jcl-over-slf4j-1.7.24.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/jetty-alpn-client-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/jetty-alpn-java-client-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/jetty-client-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/jetty-http-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/jetty-io-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/jetty-util-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/noggit-0.8.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/slf4j-api-1.7.24.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/stax2-api-3.1.4.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/solrj/lib/zookeeper-3.4.14.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/http2-common-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/http2-hpack-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/http2-server-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/javax.servlet-api-3.1.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-alpn-java-server-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-alpn-server-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-continuation-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-deploy-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-http-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-io-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-jmx-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-rewrite-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-security-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-server-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-servlet-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-servlets-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-util-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-webapp-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/jetty-xml-9.4.14.v20181114.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/metrics-core-4.0.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/metrics-graphite-4.0.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/metrics-jetty9-4.0.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/metrics-jmx-4.0.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/server/lib/metrics-jvm-4.0.5.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/example/example-DIH/solr/db/lib/hsqldb-2.4.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/core/classes/java:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/test-framework/lib/hamcrest-core-1.3.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/test-framework/lib/junit-4.12.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/test-framework/lib/randomizedtesting-runner-2.7.2.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/antlr4-runtime-4.5.1-1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/byte-buddy-1.9.3.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/commons-logging-1.1.3.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/commons-text-1.6.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/hadoop-common-3.2.0-tests.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/hadoop-hdfs-3.2.0-tests.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/hadoop-hdfs-3.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/hadoop-minicluster-3.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/hadoop-minikdc-3.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/jersey-core-1.19.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/jersey-server-1.19.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/jersey-servlet-1.19.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerb-admin-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerb-client-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerb-common-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerb-crypto-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerb-identity-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerb-server-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerb-simplekdc-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerby-config-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerby-kdc-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/kerby-util-1.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/mockito-core-2.23.4.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/netty-all-4.0.52.Final.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/test-lib/objenesis-2.6.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/analysis/icu/lucene-analyzers-icu-8.1.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/contrib/solr-analysis-extras/classes/java:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/queryparser/classes/test:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/build/backward-codecs/classes/test:/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/contrib/analysis-extras/lib/icu4j-62.1.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/Users/jenkins/.ant/lib/ivy-2.4.0.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/Users/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.7.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20190423_124430_50813314598490406613459.events @/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20190423_124430_5087238838297494535032.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  See process stdout at: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20190423_124430_5086724720299292625791.sysout
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1542)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:123)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:997)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:994)
   [junit4] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   [junit4] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
   [junit4] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4] 	at java.base/java.lang.Thread.run(Thread.java:844)

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/build.xml:634: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/build.xml:578: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/build.xml:59: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build.xml:267: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/common-build.xml:558: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/common-build.xml:1573: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/lucene/common-build.xml:1100: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  See process stdout at: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20190423_124430_5086724720299292625791.sysout

Total time: 114 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-8.x-MacOSX (64bit/jdk-9) - Build # 98 - Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/98/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.PeerSyncReplicationTest.test

Error Message:
expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: expected:<0> but was:<1>
	at __randomizedtesting.SeedInfo.seed([9F88C241A0F13D3B:17DCFD9B0E0D50C3]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:645)
	at org.junit.Assert.assertEquals(Assert.java:631)
	at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:202)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 13063 lines...]
   [junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/init-core-data-001
   [junit4]   2> 401787 INFO  (SUITE-PeerSyncReplicationTest-seed#[9F88C241A0F13D3B]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 401795 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer Overriding violation report action to: IGNORE
   [junit4]   2> 401795 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 401795 INFO  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 401796 INFO  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 401901 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer start zk server on port:49798
   [junit4]   2> 401901 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:49798
   [junit4]   2> 401901 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 49798
   [junit4]   2> 401946 INFO  (zkConnectionManagerCallback-2441-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 401952 INFO  (zkConnectionManagerCallback-2443-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 401955 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 401957 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 401959 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 401960 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 401962 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 401967 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 401970 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 401972 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 401974 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 401976 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 401978 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer put /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 401980 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 402478 WARN  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 402479 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 402479 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 402479 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 9+181
   [junit4]   2> 402481 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 402481 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 402481 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 402482 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@77b4628a{/,null,AVAILABLE}
   [junit4]   2> 402488 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@694d8105{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:49802}
   [junit4]   2> 402488 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.s.Server Started @402587ms
   [junit4]   2> 402489 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/tempDir-001/control/data, hostPort=49802, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/control-001/cores, replicaType=NRT}
   [junit4]   2> 402489 ERROR (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 402489 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 402489 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.1.0
   [junit4]   2> 402489 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 402489 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 402490 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-04-23T22:51:07.271015Z
   [junit4]   2> 402493 INFO  (zkConnectionManagerCallback-2445-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 402495 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 402496 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/control-001/solr.xml
   [junit4]   2> 402499 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 402499 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 402500 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 402804 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 402806 WARN  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@7c5499[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 402810 WARN  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@537e6335[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 402811 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49798/solr
   [junit4]   2> 402822 INFO  (zkConnectionManagerCallback-2452-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 402832 INFO  (zkConnectionManagerCallback-2454-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 403040 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49802_
   [junit4]   2> 403041 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.c.Overseer Overseer (id=72074777024856068-127.0.0.1:49802_-n_0000000000) starting
   [junit4]   2> 403057 INFO  (zkConnectionManagerCallback-2461-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 403068 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49798/solr ready
   [junit4]   2> 403070 INFO  (OverseerStateUpdate-72074777024856068-127.0.0.1:49802_-n_0000000000) [n:127.0.0.1:49802_    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:49802_
   [junit4]   2> 403070 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49802_
   [junit4]   2> 403072 INFO  (OverseerStateUpdate-72074777024856068-127.0.0.1:49802_-n_0000000000) [n:127.0.0.1:49802_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 403086 INFO  (zkCallback-2460-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 403188 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 403237 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 403279 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 403279 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 403280 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [n:127.0.0.1:49802_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/control-001/cores
   [junit4]   2> 403328 INFO  (zkConnectionManagerCallback-2467-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 403329 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 403333 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49798/solr ready
   [junit4]   2> 403338 INFO  (qtp808325440-6141) [n:127.0.0.1:49802_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:49802_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 403344 INFO  (OverseerThreadFactory-1580-thread-1-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 403472 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_    x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 403472 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_    x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 404502 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
   [junit4]   2> 404528 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 404690 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 404718 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 404719 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 404730 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 404738 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=61.6689453125, floorSegmentMB=0.8349609375, forceMergeDeletesPctAllowed=23.53192282996779, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18164989656845165, deletesPctAllowed=22.03075618366135
   [junit4]   2> 404770 WARN  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 404850 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 404850 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 404853 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 404853 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 404854 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=36.6171875, floorSegmentMB=0.8720703125, forceMergeDeletesPctAllowed=29.34821112636855, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=32.6752313844925
   [junit4]   2> 404876 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2a55e32e[control_collection_shard1_replica_n1] main]
   [junit4]   2> 404882 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 404883 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 404883 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 404884 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1631647033893847040
   [junit4]   2> 404888 INFO  (searcherExecutor-1585-thread-1-processing-n:127.0.0.1:49802_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@2a55e32e[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 404893 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 404893 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
   [junit4]   2> 404902 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 404902 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 404902 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49802/control_collection_shard1_replica_n1/
   [junit4]   2> 404902 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 404903 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:49802/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 404903 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/72074777024856068-core_node2-n_0000000000
   [junit4]   2> 404906 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49802/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 405012 INFO  (zkCallback-2453-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 405012 INFO  (zkCallback-2453-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 405016 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 405025 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1553
   [junit4]   2> 405041 INFO  (qtp808325440-6141) [n:127.0.0.1:49802_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 405121 INFO  (zkCallback-2453-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 405121 INFO  (zkCallback-2453-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 405121 INFO  (zkCallback-2453-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 405122 INFO  (qtp808325440-6141) [n:127.0.0.1:49802_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:49802_&wt=javabin&version=2} status=0 QTime=1784
   [junit4]   2> 405130 INFO  (zkConnectionManagerCallback-2473-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 405132 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 405134 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49798/solr ready
   [junit4]   2> 405134 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 405136 INFO  (qtp808325440-6142) [n:127.0.0.1:49802_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 405146 INFO  (OverseerThreadFactory-1580-thread-2-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 405146 INFO  (OverseerCollectionConfigSetProcessor-72074777024856068-127.0.0.1:49802_-n_0000000000) [n:127.0.0.1:49802_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 405366 WARN  (OverseerThreadFactory-1580-thread-2-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 405369 INFO  (qtp808325440-6142) [n:127.0.0.1:49802_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 405370 INFO  (qtp808325440-6142) [n:127.0.0.1:49802_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=234
   [junit4]   2> 405371 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=3
   [junit4]   2> 405735 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-1-001 of type NRT
   [junit4]   2> 405737 WARN  (closeThreadPool-2474-thread-1) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 405737 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 405737 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 405737 INFO  (closeThreadPool-2474-thread-1) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 9+181
   [junit4]   2> 405739 INFO  (closeThreadPool-2474-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 405739 INFO  (closeThreadPool-2474-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 405739 INFO  (closeThreadPool-2474-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 405740 INFO  (closeThreadPool-2474-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@138399b2{/,null,AVAILABLE}
   [junit4]   2> 405741 INFO  (closeThreadPool-2474-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4bcb7286{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:49812}
   [junit4]   2> 405741 INFO  (closeThreadPool-2474-thread-1) [    ] o.e.j.s.Server Started @405839ms
   [junit4]   2> 405741 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/tempDir-001/jetty1, hostPort=49812, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-1-001/cores, replicaType=NRT}
   [junit4]   2> 405741 ERROR (closeThreadPool-2474-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 405741 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 405742 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.1.0
   [junit4]   2> 405742 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 405742 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 405742 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-04-23T22:51:10.523114Z
   [junit4]   2> 405759 INFO  (zkConnectionManagerCallback-2476-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 405764 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 405764 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-1-001/solr.xml
   [junit4]   2> 405798 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 405798 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 405799 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 406180 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 406182 WARN  (closeThreadPool-2474-thread-1) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@27e7df26[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 406186 WARN  (closeThreadPool-2474-thread-1) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@cebdf8e[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 406187 INFO  (closeThreadPool-2474-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49798/solr
   [junit4]   2> 406191 INFO  (zkConnectionManagerCallback-2483-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 406202 INFO  (zkConnectionManagerCallback-2485-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 406212 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 406224 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.c.ZkController Publish node=127.0.0.1:49812_ as DOWN
   [junit4]   2> 406226 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 406227 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49812_
   [junit4]   2> 406229 INFO  (zkCallback-2453-thread-3) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 406230 INFO  (zkCallback-2460-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 406230 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 406230 INFO  (zkCallback-2484-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 406235 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-2-001 of type NRT
   [junit4]   2> 406236 WARN  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 406237 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 406237 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 406237 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 9+181
   [junit4]   2> 406238 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 406238 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 406238 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 406239 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@589e5906{/,null,AVAILABLE}
   [junit4]   2> 406239 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@57b35b7d{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:49816}
   [junit4]   2> 406239 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.Server Started @406338ms
   [junit4]   2> 406239 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/tempDir-001/jetty2, hostPort=49816, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-2-001/cores, replicaType=NRT}
   [junit4]   2> 406240 ERROR (closeThreadPool-2474-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 406240 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 406240 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.1.0
   [junit4]   2> 406240 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 406240 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 406240 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-04-23T22:51:11.021601Z
   [junit4]   2> 406244 INFO  (zkConnectionManagerCallback-2489-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 406245 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 406245 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-2-001/solr.xml
   [junit4]   2> 406249 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 406249 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 406266 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 406334 INFO  (zkConnectionManagerCallback-2495-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 406337 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 406340 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49798/solr ready
   [junit4]   2> 406341 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 406386 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 406388 WARN  (closeThreadPool-2474-thread-2) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@5ea11cc2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 406392 WARN  (closeThreadPool-2474-thread-2) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@7b12511c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 406393 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49798/solr
   [junit4]   2> 406397 INFO  (zkConnectionManagerCallback-2502-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 406419 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 406430 INFO  (zkConnectionManagerCallback-2504-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 406445 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 406445 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 406454 INFO  (closeThreadPool-2474-thread-1) [n:127.0.0.1:49812_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-1-001/cores
   [junit4]   2> 406455 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 406470 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.c.ZkController Publish node=127.0.0.1:49816_ as DOWN
   [junit4]   2> 406473 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 406473 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49816_
   [junit4]   2> 406476 INFO  (zkCallback-2484-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 406476 INFO  (zkCallback-2460-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 406476 INFO  (zkCallback-2453-thread-3) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 406477 INFO  (zkCallback-2494-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 406477 INFO  (zkCallback-2503-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 406485 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 406514 INFO  (zkConnectionManagerCallback-2511-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 406516 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 406530 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49798/solr ready
   [junit4]   2> 406531 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 406601 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 406666 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 406666 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 406669 INFO  (closeThreadPool-2474-thread-2) [n:127.0.0.1:49816_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-2-001/cores
   [junit4]   2> 407108 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-3-001 of type NRT
   [junit4]   2> 407109 WARN  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 407110 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 407110 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 407110 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 9+181
   [junit4]   2> 407111 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 407111 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 407112 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 407112 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e1b39d0{/,null,AVAILABLE}
   [junit4]   2> 407113 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@8eb18d8{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:49822}
   [junit4]   2> 407114 INFO  (closeThreadPool-2474-thread-2) [    ] o.e.j.s.Server Started @407212ms
   [junit4]   2> 407114 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/tempDir-001/jetty3, hostPort=49822, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-3-001/cores}
   [junit4]   2> 407115 ERROR (closeThreadPool-2474-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 407115 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 407115 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.1.0
   [junit4]   2> 407115 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 407115 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 407115 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-04-23T22:51:11.896402Z
   [junit4]   2> 407126 INFO  (zkConnectionManagerCallback-2514-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 407128 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 407128 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-3-001/solr.xml
   [junit4]   2> 407132 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 407132 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 407133 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 407160 INFO  (OverseerCollectionConfigSetProcessor-72074777024856068-127.0.0.1:49802_-n_0000000000) [n:127.0.0.1:49802_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 407493 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 407495 WARN  (closeThreadPool-2474-thread-2) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@3f753eda[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 407500 WARN  (closeThreadPool-2474-thread-2) [    ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@39d776d[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 407502 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49798/solr
   [junit4]   2> 407510 INFO  (zkConnectionManagerCallback-2521-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 407524 INFO  (zkConnectionManagerCallback-2523-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 407533 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 407544 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.ZkController Publish node=127.0.0.1:49822_ as DOWN
   [junit4]   2> 407545 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 407546 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49822_
   [junit4]   2> 407549 INFO  (zkCallback-2484-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 407549 INFO  (zkCallback-2453-thread-3) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 407549 INFO  (zkCallback-2460-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 407549 INFO  (zkCallback-2472-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 407549 INFO  (zkCallback-2522-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 407549 INFO  (zkCallback-2503-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 407549 INFO  (zkCallback-2494-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 407557 INFO  (zkCallback-2510-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 407594 INFO  (zkConnectionManagerCallback-2530-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 407596 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 407598 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:49798/solr ready
   [junit4]   2> 407599 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 407622 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 407638 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 407639 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 407640 INFO  (closeThreadPool-2474-thread-2) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-3-001/cores
   [junit4]   2> 407731 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:49812_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 407742 INFO  (OverseerThreadFactory-1580-thread-3-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:49812_ for creating new replica of shard shard1 for collection collection1
   [junit4]   2> 407745 INFO  (qtp808325440-6145) [n:127.0.0.1:49802_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:49816_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 407748 INFO  (OverseerThreadFactory-1580-thread-3-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 407755 INFO  (qtp808325440-6140) [n:127.0.0.1:49802_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:49822_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 407763 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_    x:collection1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 408792 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
   [junit4]   2> 408824 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 408947 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 408996 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
   [junit4]   2> 408997 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 408997 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-1-001/cores/collection1_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-1-001/cores/collection1_shard1_replica_n1/data/]
   [junit4]   2> 409002 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=61.6689453125, floorSegmentMB=0.8349609375, forceMergeDeletesPctAllowed=23.53192282996779, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18164989656845165, deletesPctAllowed=22.03075618366135
   [junit4]   2> 409084 WARN  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 409165 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 409166 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 409169 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 409169 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 409170 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=36.6171875, floorSegmentMB=0.8720703125, forceMergeDeletesPctAllowed=29.34821112636855, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=32.6752313844925
   [junit4]   2> 409171 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7a68c8ee[collection1_shard1_replica_n1] main]
   [junit4]   2> 409173 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 409174 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 409174 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 409174 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1631647038392238080
   [junit4]   2> 409179 INFO  (searcherExecutor-1617-thread-1-processing-n:127.0.0.1:49812_ x:collection1_shard1_replica_n1 c:collection1 s:shard1) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@7a68c8ee[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 409196 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 409196 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
   [junit4]   2> 409217 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 409217 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 409217 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49812/collection1_shard1_replica_n1/
   [junit4]   2> 409223 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 409225 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:49812/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 409225 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72074777024856074-core_node2-n_0000000000
   [junit4]   2> 409234 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49812/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 409342 INFO  (zkCallback-2484-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 409343 INFO  (zkCallback-2484-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 409350 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 409370 INFO  (qtp1551035458-6204) [n:127.0.0.1:49812_ c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1608
   [junit4]   2> 409376 INFO  (qtp808325440-6143) [n:127.0.0.1:49802_ c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:49812_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1645
   [junit4]   2> 409459 INFO  (zkCallback-2484-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 409459 INFO  (zkCallback-2484-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 409460 INFO  (zkCallback-2484-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 409774 INFO  (OverseerCollectionConfigSetProcessor-72074777024856068-127.0.0.1:49802_-n_0000000000) [n:127.0.0.1:49802_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 409779 INFO  (OverseerThreadFactory-1580-thread-4-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:49816_ for creating new replica of shard shard1 for collection collection1
   [junit4]   2> 409786 INFO  (OverseerThreadFactory-1580-thread-4-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 409798 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_    x:collection1_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 409915 INFO  (zkCallback-2484-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 409915 INFO  (zkCallback-2484-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 409915 INFO  (zkCallback-2484-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 410836 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
   [junit4]   2> 410893 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.s.IndexSchema [collection1_shard1_replica_n3] Schema name=test
   [junit4]   2> 411064 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 411106 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n3' using configuration from collection collection1, trusted=true
   [junit4]   2> 411106 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n3' (registry 'solr.core.collection1.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f4a50fc
   [junit4]   2> 411107 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.SolrCore [[collection1_shard1_replica_n3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-2-001/cores/collection1_shard1_replica_n3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001/shard-2-001/cores/collection1_shard1_replica_n3/data/]
   [junit4]   2> 411114 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=61.6689453125, floorSegmentMB=0.8349609375, forceMergeDeletesPctAllowed=23.53192282996779, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18164989656845165, deletesPctAllowed=22.03075618366135
   [junit4]   2> 411180 WARN  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 411395 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 411395 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 411401 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 411401 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 411403 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=36.6171875, floorSegmentMB=0.8720703125, forceMergeDeletesPctAllowed=29.34821112636855, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=32.6752313844925
   [junit4]   2> 411433 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@5274c1b5[collection1_shard1_replica_n3] main]
   [junit4]   2> 411435 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 411435 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 411436 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 411436 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1631647040764116992
   [junit4]   2> 411442 INFO  (searcherExecutor-1622-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.SolrCore [collection1_shard1_replica_n3] Registered new searcher Searcher@5274c1b5[collection1_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 411460 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
   [junit4]   2> 411460 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
   [junit4]   2> 411478 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n3
   [junit4]   2> 411478 INFO  (updateExecutor-2498-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 411479 INFO  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 411479 INFO  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy startupVersions is empty
   [junit4]   2> 411491 INFO  (qtp1551035458-6206) [n:127.0.0.1:49812_ c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp= path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=6
   [junit4]   2> 411491 INFO  (qtp1551035458-6206) [n:127.0.0.1:49812_ c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request [collection1_shard1_replica_n1]  webapp= path=/admin/ping params={wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 411491 INFO  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n3]
   [junit4]   2> 411492 INFO  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 411492 INFO  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_n3] as recovering, leader is [http://127.0.0.1:49812/collection1_shard1_replica_n1/] and I am [http://127.0.0.1:49816/collection1_shard1_replica_n3/]
   [junit4]   2> 411496 INFO  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:49812]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_n1&nodeName=127.0.0.1:49816_&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 411502 INFO  (qtp1551035458-6207) [n:127.0.0.1:49812_    x:collection1_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 411502 INFO  (qtp1551035458-6207) [n:127.0.0.1:49812_    x:collection1_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:49816_, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1_shard1_replica_n3","base_url":"http://127.0.0.1:49816","node_name":"127.0.0.1:49816_","state":"down","type":"NRT"}
   [junit4]   2> 411514 INFO  (qtp594823335-6229) [n:127.0.0.1:49816_ c:collection1 s:shard1  x:collection1_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1716
   [junit4]   2> 411526 INFO  (qtp808325440-6145) [n:127.0.0.1:49802_ c:collection1   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:49816_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=3781
   [junit4]   2> 411608 INFO  (zkCallback-2484-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411608 INFO  (zkCallback-2484-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411608 INFO  (zkCallback-2484-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411609 INFO  (watches-2486-thread-3) [    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:49816_, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1_shard1_replica_n3","base_url":"http://127.0.0.1:49816","node_name":"127.0.0.1:49816_","state":"recovering","type":"NRT"}
   [junit4]   2> 411609 INFO  (qtp1551035458-6207) [n:127.0.0.1:49812_    x:collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:49816_&onlyIfLeaderActive=true&core=collection1_shard1_replica_n1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=109
   [junit4]   2> 411610 INFO  (zkCallback-2503-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411610 INFO  (zkCallback-2503-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411611 INFO  (zkCallback-2503-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411611 INFO  (zkCallback-2503-thread-4) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411792 INFO  (OverseerCollectionConfigSetProcessor-72074777024856068-127.0.0.1:49802_-n_0000000000) [n:127.0.0.1:49802_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 411794 INFO  (OverseerThreadFactory-1580-thread-5-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:49822_ for creating new replica of shard shard1 for collection collection1
   [junit4]   2> 411800 INFO  (OverseerThreadFactory-1580-thread-5-processing-n:127.0.0.1:49802_) [n:127.0.0.1:49802_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 411813 INFO  (qtp206095102-6271) [n:127.0.0.1:49822_    x:collection1_shard1_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n5&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 411923 INFO  (zkCallback-2484-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411923 INFO  (zkCallback-2503-thread-4) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411923 INFO  (zkCallback-2484-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411923 INFO  (zkCallback-2484-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411923 INFO  (zkCallback-2503-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411924 INFO  (zkCallback-2503-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 411924 INFO  (zkCallback-2503-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 412117 INFO  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:49812/collection1_shard1_replica_n1/] - recoveringAfterStartup=[true]
   [junit4]   2> 412117 WARN  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates
   [junit4]   2> 412117 INFO  (recoveryExecutor-2500-thread-1-processing-n:127.0.0.1:49816_ x:collection1_shard1_replica_n3 c:collection1 s:shard1 r:core_node4) [n:127.0.0.1:49816_ c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.

[...truncated too long message...]

",
   [junit4]   1>                     "type":"NRT",
   [junit4]   1>                     "leader":"true"}}}}}}
   [junit4]   1>     /solr/collections/control_collection/terms (1)
   [junit4]   1>      /solr/collections/control_collection/terms/shard1 (0)
   [junit4]   1>      DATA:
   [junit4]   1>          {"core_node2":1}
   [junit4]   1>     /solr/collections/control_collection/leader_elect (1)
   [junit4]   1>      /solr/collections/control_collection/leader_elect/shard1 (1)
   [junit4]   1>       /solr/collections/control_collection/leader_elect/shard1/election (0)
   [junit4]   1>   /solr/live_nodes (0)
   [junit4]   1>   /solr/overseer_elect (1)
   [junit4]   1>    /solr/overseer_elect/election (0)
   [junit4]   1>   /solr/security.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {}
   [junit4]   1>   /solr/clusterstate.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {}
   [junit4]   1>   /solr/autoscaling (4)
   [junit4]   1>    /solr/autoscaling/nodeAdded (0)
   [junit4]   1>    /solr/autoscaling/nodeLost (3)
   [junit4]   1>     /solr/autoscaling/nodeLost/127.0.0.1:49816_ (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {"timestamp":1556059885511842802}
   [junit4]   1>     /solr/autoscaling/nodeLost/127.0.0.1:49812_ (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {"timestamp":1556059907145926876}
   [junit4]   1>     /solr/autoscaling/nodeLost/127.0.0.1:49822_ (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {"timestamp":1556059885457855290}
   [junit4]   1>    /solr/autoscaling/triggerState (2)
   [junit4]   1>     /solr/autoscaling/triggerState/.scheduled_maintenance (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {"lastRunAt":1556059867850}
   [junit4]   1>     /solr/autoscaling/triggerState/.auto_add_replicas (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {
   [junit4]   1>           "lastLiveNodes":[
   [junit4]   1>             "127.0.0.1:49802_",
   [junit4]   1>             "127.0.0.1:49816_",
   [junit4]   1>             "127.0.0.1:49822_"],
   [junit4]   1>           "nodeNameVsTimeRemoved":{}}
   [junit4]   1>    /solr/autoscaling/events (2)
   [junit4]   1>     /solr/autoscaling/events/.scheduled_maintenance (0)
   [junit4]   1>     /solr/autoscaling/events/.auto_add_replicas (0)
   [junit4]   1>   /solr/autoscaling.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {
   [junit4]   1>         "triggers":{
   [junit4]   1>           ".auto_add_replicas":{
   [junit4]   1>             "name":".auto_add_replicas",
   [junit4]   1>             "event":"nodeLost",
   [junit4]   1>             "waitFor":10,
   [junit4]   1>             "enabled":true,
   [junit4]   1>             "actions":[
   [junit4]   1>               {
   [junit4]   1>                 "name":"auto_add_replicas_plan",
   [junit4]   1>                 "class":"solr.AutoAddReplicasPlanAction"},
   [junit4]   1>               {
   [junit4]   1>                 "name":"execute_plan",
   [junit4]   1>                 "class":"solr.ExecutePlanAction"}]},
   [junit4]   1>           ".scheduled_maintenance":{
   [junit4]   1>             "name":".scheduled_maintenance",
   [junit4]   1>             "event":"scheduled",
   [junit4]   1>             "startTime":"NOW",
   [junit4]   1>             "every":"+1DAY",
   [junit4]   1>             "enabled":true,
   [junit4]   1>             "actions":[
   [junit4]   1>               {
   [junit4]   1>                 "name":"inactive_shard_plan",
   [junit4]   1>                 "class":"solr.InactiveShardPlanAction"},
   [junit4]   1>               {
   [junit4]   1>                 "name":"inactive_markers_plan",
   [junit4]   1>                 "class":"solr.InactiveMarkersPlanAction"},
   [junit4]   1>               {
   [junit4]   1>                 "name":"execute_plan",
   [junit4]   1>                 "class":"solr.ExecutePlanAction"}]}},
   [junit4]   1>         "listeners":{
   [junit4]   1>           ".auto_add_replicas.system":{
   [junit4]   1>             "beforeAction":[],
   [junit4]   1>             "afterAction":[],
   [junit4]   1>             "stage":[
   [junit4]   1>               "STARTED",
   [junit4]   1>               "ABORTED",
   [junit4]   1>               "SUCCEEDED",
   [junit4]   1>               "FAILED",
   [junit4]   1>               "BEFORE_ACTION",
   [junit4]   1>               "AFTER_ACTION",
   [junit4]   1>               "IGNORED"],
   [junit4]   1>             "trigger":".auto_add_replicas",
   [junit4]   1>             "class":"org.apache.solr.cloud.autoscaling.SystemLogListener"},
   [junit4]   1>           ".scheduled_maintenance.system":{
   [junit4]   1>             "beforeAction":[],
   [junit4]   1>             "afterAction":[],
   [junit4]   1>             "stage":[
   [junit4]   1>               "STARTED",
   [junit4]   1>               "ABORTED",
   [junit4]   1>               "SUCCEEDED",
   [junit4]   1>               "FAILED",
   [junit4]   1>               "BEFORE_ACTION",
   [junit4]   1>               "AFTER_ACTION",
   [junit4]   1>               "IGNORED"],
   [junit4]   1>             "trigger":".scheduled_maintenance",
   [junit4]   1>             "class":"org.apache.solr.cloud.autoscaling.SystemLogListener"}},
   [junit4]   1>         "properties":{}}
   [junit4]   1>   /solr/clusterprops.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {"legacyCloud":"true"}
   [junit4]   1> 
   [junit4]   2> 467428 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 467642 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:49798
   [junit4]   2> 467642 INFO  (TEST-PeerSyncReplicationTest.test-seed#[9F88C241A0F13D3B]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 49798
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=PeerSyncReplicationTest -Dtests.method=test -Dtests.seed=9F88C241A0F13D3B -Dtests.slow=true -Dtests.locale=es-AR -Dtests.timezone=America/Chicago -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 65.9s J0 | PeerSyncReplicationTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<0> but was:<1>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F88C241A0F13D3B:17DCFD9B0E0D50C3]:0)
   [junit4]    > 	at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:202)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-8.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9F88C241A0F13D3B-001
   [junit4]   2> Apr 23, 2019 10:52:12 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@342dedd7), locale=es-AR, timezone=America/Chicago
   [junit4]   2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 9 (64-bit)/cpus=3,threads=1,free=167864208,total=418906112
   [junit4]   2> NOTE: All tests run in this JVM: [TestSizeLimitedDistributedMap, QueryEqualityTest, TestCloudManagedSchema, TestSolrCloudWithDelegationTokens, CreateCollectionCleanupTest, SyncSliceTest, LoggingHandlerTest, FieldMutatingUpdateProcessorTest, OverseerRolesTest, AnalyticsMergeStrategyTest, TestSchemaManager, TestSolrConfigHandlerCloud, LeaderTragicEventTest, TestExportWriter, TestQueryUtils, ExplicitHLLTest, HttpPartitionWithTlogReplicasTest, RAMDirectoryFactoryTest, TestManagedSchemaAPI, TestCloudNestedDocsSort, SSLMigrationTest, SoftAutoCommitTest, TestJsonRequest, TestSolrCloudWithSecureImpersonation, MetricTriggerIntegrationTest, SolrSlf4jReporterTest, SpellPossibilityIteratorTest, PeerSyncReplicationTest]
   [junit4] Completed [81/854 (1!)] on J0 in 66.66s, 1 test, 1 failure <<< FAILURES!

[...truncated 45820 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/98/consoleText

[repro] Revision: 0cfd85baef7f6f6fb997330b9a14471d66a62889

[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC"
[repro] Repro line:  ant test  -Dtestcase=PeerSyncReplicationTest -Dtests.method=test -Dtests.seed=9F88C241A0F13D3B -Dtests.slow=true -Dtests.locale=es-AR -Dtests.timezone=America/Chicago -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       PeerSyncReplicationTest
[repro] ant compile-test

[...truncated 3603 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.PeerSyncReplicationTest" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=9F88C241A0F13D3B -Dtests.slow=true -Dtests.locale=es-AR -Dtests.timezone=America/Chicago -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 75 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.PeerSyncReplicationTest
[repro] Exiting with code 0

[...truncated 73 lines...]